A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis

24 July 2023

Hiroki Furuta

Papers citing "A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis"

50 / 160 papers shown

Title
OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web Raghav Kapoor Y. Butala M. Russak Jing Yu Koh Kiran Kamble Waseem Alshikh Ruslan Salakhutdinov LLMAG 51 44 0 27 Feb 2024
Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMs Sumedh Rasal E. Hauer 32 0 0 26 Feb 2024
Budget-Constrained Tool Learning with Planning Yuanhang Zheng Peng Li Mingshi Yan Ji Zhang Fei Huang Yang Janet Liu 32 3 0 25 Feb 2024
On the Multi-turn Instruction Following for Conversational Web Agents Yang Deng Xuan Zhang Wenxuan Zhang Yifei Yuan See-Kiong Ng Tat-Seng Chua LLMAG LM&Ro 31 22 0 23 Feb 2024
Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents Wenkai Yang Xiaohan Bi Yankai Lin Sishuo Chen Jie Zhou Xu Sun LLMAG AAML 38 53 0 17 Feb 2024
A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts Kuang-Huei Lee Xinyun Chen Hiroki Furuta John F. Canny Ian S. Fischer RALM 55 29 0 15 Feb 2024
UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Construction Yansong NING Hao Liu LLMAG 29 2 0 10 Feb 2024
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue Xing Han Lù Zdeněk Kasner Siva Reddy 30 59 0 08 Feb 2024
Dual-View Visual Contextualization for Web Navigation Jihyung Kil Chan Hee Song Boyuan Zheng Xiang Deng Yu-Chuan Su Wei-Lun Chao EgoV 22 12 0 06 Feb 2024
tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs) Junhua Zeng Chao Li Zhun Sun Qibin Zhao Guoxu Zhou 37 4 0 04 Feb 2024
AMOR: A Recipe for Building Adaptable Modular Knowledge Agents Through Process Feedback Jian-Yu Guan Wei Yu Wu Zujie Wen Peng Xu Hongning Wang Minlie Huang LRM 21 16 0 02 Feb 2024
SwarmBrain: Embodied agent for real-time strategy game StarCraft II via large language models Xiao Shao Weifu Jiang Fei Zuo Mengqing Liu LLMAG 31 7 0 31 Jan 2024
Investigate-Consolidate-Exploit: A General Strategy for Inter-Task Agent Self-Evolution Cheng Qian Shihao Liang Yujia Qin Yining Ye Xin Cong Yankai Lin Yesai Wu Zhiyuan Liu Maosong Sun LLMAG 24 12 0 25 Jan 2024
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models Hongliang He Wenlin Yao Kaixin Ma Wenhao Yu Yong Dai Hongming Zhang Zhenzhong Lan Dong Yu LLMAG 38 121 0 25 Jan 2024
VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks Jing Yu Koh Robert Lo Lawrence Jang Vikram Duvvur Ming Chong Lim Po-Yu Huang Graham Neubig Shuyan Zhou Ruslan Salakhutdinov Daniel Fried 23 0 0 24 Jan 2024
SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents Kanzhi Cheng Qiushi Sun Yougang Chu Fangzhi Xu Yantao Li Jianbing Zhang Zhiyong Wu LLMAG 172 138 0 17 Jan 2024
MobileAgent: enhancing mobile control via human-machine interaction and SOP integration Tinghe Ding LLMAG LM&Ro 36 6 0 04 Jan 2024
GPT-4V(ision) is a Generalist Web Agent, if Grounded Boyuan Zheng Boyu Gou Jihyung Kil Huan Sun Yu-Chuan Su MLLM VLM LLMAG 46 207 0 03 Jan 2024
A Simple LLM Framework for Long-Range Video Question-Answering Ce Zhang Taixi Lu Md. Mohaiminul Islam Ziyang Wang Shoubin Yu Mohit Bansal Gedas Bertasius 105 80 0 28 Dec 2023
AppAgent: Multimodal Agents as Smartphone Users C. Zhang Zhao Yang Jiaxuan Liu Yucheng Han Xin Chen Zebiao Huang Bin-Bin Fu Gang Yu LM&Ro LLMAG 24 158 0 21 Dec 2023
Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives Chen Gao Xiaochong Lan Nian Li Yuan Yuan Jingtao Ding Zhilun Zhou Fengli Xu Yong Li LLMAG AI4CE LM&Ro 41 103 0 19 Dec 2023
Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents Zhuosheng Zhang Yao Yao Aston Zhang Xiangru Tang Xinbei Ma ... Yiming Wang Mark B. Gerstein Rui Wang Gongshen Liu Hai Zhao LLMAG LM&Ro LRM 36 53 0 20 Nov 2023
Testing Language Model Agents Safely in the Wild Silen Naihin David Atkinson Marc Green Merwane Hamadi Craig Swift Douglas Schonholtz Adam Tauman Kalai David Bau LLMAG 29 19 0 17 Nov 2023
AllTogether: Investigating the Efficacy of Spliced Prompt for Web Navigation using Large Language Models Jiarun Liu Wentao Hu Chunhong Zhang 22 2 0 20 Oct 2023
GestureGPT: Toward Zero-shot Interactive Gesture Understanding and Grounding with Large Language Model Agents Xin Zeng Xiaoyu Wang Tengxiang Zhang Chun Yu Shengdong Zhao Yiqiang Chen LLMAG LM&Ro SLR 19 1 0 19 Oct 2023
A Zero-Shot Language Agent for Computer Control with Structured Reflection Tao Li Gang Li Zhiwei Deng Bryan Wang Yang Li LM&Ro LLMAG 57 23 0 12 Oct 2023
Formally Specifying the High-Level Behavior of LLM-Based Agents M. Crouse Ibrahim Abdelaziz Ramón Fernández Astudillo Kinjal Basu Soham Dan Sadhana Kumaravel Achille Fokoue Pavan Kapanipathi Salim Roukos Luis A. Lastras LLMAG 18 8 0 12 Oct 2023
Lemur: Harmonizing Natural Language and Code for Language Agents Yiheng Xu Hongjin Su Chen Xing Boyu Mi Qian Liu ... Siheng Zhao Lingpeng Kong Bailin Wang Caiming Xiong Tao Yu 32 67 0 10 Oct 2023
How FaR Are Large Language Models From Agents with Theory-of-Mind? Pei Zhou Aman Madaan Srividya Pranavi Potharaju Aditya Gupta Kevin R. McKee ... Xiang Ren Swaroop Mishra Aida Nematzadeh Shyam Upadhyay Manaal Faruqui LRM AI4CE 22 48 0 04 Oct 2023
AXNav: Replaying Accessibility Tests from Natural Language Maryam Taeb Amanda Swearngin E. Schoop Ruijia Cheng Yue Jiang Jeffrey Nichols 26 37 0 03 Oct 2023
Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation Shenzhi Wang Chang Liu Zilong Zheng Siyuan Qi Shuo Chen Qisen Yang Andrew Zhao Chaofei Wang Shiji Song Gao Huang LLMAG 31 62 0 02 Oct 2023
Identifying the Risks of LM Agents with an LM-Emulated Sandbox Yangjun Ruan Honghua Dong Andrew Wang Silviu Pitis Yongchao Zhou Jimmy Ba Yann Dubois Chris J. Maddison Tatsunori Hashimoto LLMAG ELM 14 97 0 25 Sep 2023
LLMR: Real-time Prompting of Interactive Worlds using Large Language Models Fernanda De La Torre Cathy Mengying Fang Han Huang Andrzej Banburski-Fahey Judith Amores Fernandez Jaron Lanier 40 45 0 21 Sep 2023
You Only Look at Screens: Multimodal Chain-of-Action Agents Zhuosheng Zhang Aston Zhang LLMAG LM&Ro 15 101 0 20 Sep 2023
Generative AI vs. AGI: The Cognitive Strengths and Weaknesses of Modern LLMs Ben Goertzel 30 13 0 19 Sep 2023
LASER: LLM Agent with State-Space Exploration for Web Navigation Kaixin Ma Hongming Zhang Hongwei Wang Xiaoman Pan Wenhao Yu Dong Yu LLMAG 24 39 0 15 Sep 2023
Agents: An Open-source Framework for Autonomous Language Agents Wangchunshu Zhou Yuchen Eleanor Jiang Long Li Jialong Wu Tiannan Wang ... Xiangru Tang Ningyu Zhang Huajun Chen Peng Cui Mrinmaya Sachan LLMAG LM&Ro AI4CE 31 87 0 14 Sep 2023
Cognitive Architectures for Language Agents T. Sumers Shunyu Yao Karthik Narasimhan Thomas L. Griffiths LLMAG LM&Ro 45 152 0 05 Sep 2023
ExpeL: LLM Agents Are Experiential Learners Andrew Zhao Daniel Huang Quentin Xu Matthieu Lin Y. Liu Gao Huang LLMAG 22 193 0 20 Aug 2023
Large Language Models for Information Retrieval: A Survey Yutao Zhu Huaying Yuan Shuting Wang Jiongnan Liu Wenhan Liu Chenlong Deng Haonan Chen Zhicheng Dou Ji-Rong Wen KELM 54 284 0 14 Aug 2023
BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents Zhiwei Liu Weiran Yao Jianguo Zhang Le Xue Shelby Heinecke ... Ran Xu P. Mùi Haiquan Wang Caiming Xiong Silvio Savarese LLMAG 29 82 0 11 Aug 2023
Gentopia: A Collaborative Platform for Tool-Augmented LLMs Binfeng Xu Xukun Liu Hua Shen Zeyu Han Yuhan Li Murong Yue Zhi-Ping Peng Yuchen Liu Ziyu Yao Dongkuan Xu LLMAG 22 19 0 08 Aug 2023
WebArena: A Realistic Web Environment for Building Autonomous Agents Shuyan Zhou Frank F. Xu Hao Zhu Xuhui Zhou Robert Lo ... Tianyue Ou Yonatan Bisk Daniel Fried Uri Alon Graham Neubig LLMAG 36 381 0 25 Jul 2023
Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control Longtao Zheng R. Wang Xinrun Wang Bo An LLMAG 22 57 0 13 Jun 2023
Multimodal Web Navigation with Instruction-Finetuned Foundation Models Hiroki Furuta Kuang-Huei Lee Ofir Nachum Yutaka Matsuo Aleksandra Faust S. Gu Izzeddin Gur LM&Ro 36 91 0 19 May 2023
Understanding HTML with Large Language Models Izzeddin Gur Ofir Nachum Yingjie Miao Mustafa Safdari Austin Huang Aakanksha Chowdhery Sharan Narang Noah Fiedel Aleksandra Faust AI4CE 138 70 0 08 Oct 2022
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan Yuan Cao LLMAG ReLM LRM 240 2,494 0 06 Oct 2022
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models Ishika Singh Valts Blukis Arsalan Mousavian Ankit Goyal Danfei Xu Jonathan Tremblay D. Fox Jesse Thomason Animesh Garg LM&Ro LLMAG 120 624 0 22 Sep 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 322 4,077 0 24 May 2022
TIE: Topological Information Enhanced Structural Reading Comprehension on Web Pages Zihan Zhao Lu Chen Ruisheng Cao Hongshen Xu Xingyu Chen Kai Yu 36 9 0 13 May 2022