Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration

24 February 2018

Papers citing "Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration"

36 / 36 papers shown

Title
VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making Jake Grigsby Yuke Zhu Michael S Ryoo Juan Carlos Niebles OffRL VLM 41 0 0 06 May 2025
Cracking the Code of Action: a Generative Approach to Affordances for Reinforcement Learning Lynn Cherif Flemming Kondrup David Venuto Ankit Anand Doina Precup Khimya Khetarpal LM&Ro 54 0 0 24 Apr 2025
REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites Divyansh Garg Shaun VanWeelden Diego Caples Andis Draguns Nikil Ravi ... Youngchul Joo Jindong Gu Charles London Christian Schroeder de Witt S. Motwani 47 1 0 15 Apr 2025
AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories Xing Han Lù Amirhossein Kazemnejad Nicholas Meade Arkil Patel Dongchan Shin Alejandra Zambrano Karolina Stañczak Peter Shaw Christopher Pal Siva Reddy LLMAG 42 1 0 11 Apr 2025
SpiritSight Agent: Advanced GUI Agent with One Look Zhiyuan Huang Ziming Cheng Junting Pan Zhaohui Hou Mingjie Zhan LLMAG 101 2 0 05 Mar 2025
Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agents Vardaan Pahuja Yadong Lu Corby Rosset Boyu Gou Arindam Mitra Spencer Whitehead Yu Su Ahmed Awadallah LLMAG LM&Ro Presented at ResearchTrend Connect \| LLMAG on 14 Mar 2025 151 3 1 20 Feb 2025
AgentStudio: A Toolkit for Building General Virtual Agents Longtao Zheng Zhiyuan Huang Zhenghai Xue Xinrun Wang Bo An Shuicheng Yan 88 14 0 17 Feb 2025
Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web Hiroki Furuta Yutaka Matsuo Aleksandra Faust Izzeddin Gur CLL 95 14 0 03 Jan 2025
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks Frank F. Xu Yufan Song Boxuan Li Yuxuan Tang Kritanjali Jain ... Wayne Chi Lawrence Jang Yiqing Xie Shuyan Zhou Graham Neubig LLMAG 129 22 0 18 Dec 2024
The BrowserGym Ecosystem for Web Agent Research Thibault Le Sellier De Chezelles Maxime Gasse Alexandre Lacoste Alexandre Drouin Massimo Caccia ... Siva Reddy Quentin Cappart Graham Neubig Ruslan Salakhutdinov Nicolas Chapados LLMAG 109 10 0 06 Dec 2024
GUI Agents with Foundation Models: A Comprehensive Survey Shuai Wang Wen Liu Jingxuan Chen Weinan Gan Xingshan Zeng ... Bin Wang Chuhan Wu Yasheng Wang Ruiming Tang Jianye Hao LLMAG 76 15 0 07 Nov 2024
Lightweight Neural App Control Filippos Christianos Georgios Papoudakis Thomas Coste Jianye Hao Jun Wang Kun Shao LM&Ro 60 4 0 23 Oct 2024
Large Language Models Empowered Personalized Web Agents Hongru Cai Yongqi Li Luu Anh Tuan Fengbin Zhu Xiaoyu Shen Wenjie Li Tat-Seng Chua LLMAG 54 12 0 22 Oct 2024
Beyond Browsing: API-Based Web Agents Yueqi Song Frank F. Xu Shuyan Zhou Graham Neubig 61 15 0 21 Oct 2024
Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation Hyungjoo Chae Namyoung Kim Kai Tzu-iunn Ong Minju Gwak Gwanwoo Song Jihoon Kim S. Kim Dongha Lee Jinyoung Yeo LLMAG 33 14 0 17 Oct 2024
ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents Ido Levy Ben Wiesel Sami Marreed Alon Oved Avi Yaeli Segev Shlomov LLMAG 37 15 0 09 Oct 2024
WebSuite: Systematically Evaluating Why Web Agents Fail Eric Li Jim Waldo LLMAG 31 5 0 01 Jun 2024
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents Christopher Rawles Sarah Clinckemaillie Yifan Chang Jonathan Waltz Gabrielle Lau ... Daniel Toyama Robert Berry Divya Tyamagundlu Timothy Lillicrap Oriana Riva LLMAG 69 44 0 23 May 2024
Benchmarking Mobile Device Control Agents across Diverse Configurations Juyong Lee Taywon Min Minyong An Changyeon Kim Kimin Lee 44 10 0 25 Apr 2024
MMInA: Benchmarking Multihop Multimodal Internet Agents Ziniu Zhang Shulin Tian Liangyu Chen Ziwei Liu LLMAG LM&Ro 35 13 0 15 Apr 2024
Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs Keen You Haotian Zhang E. Schoop Floris Weers Amanda Swearngin Jeffrey Nichols Yinfei Yang Zhe Gan MLLM 47 84 0 08 Apr 2024
Tur[k]ingBench: A Challenge Benchmark for Web Agents Kevin Xu Yeganeh Kordi Kate Sanders Yizhong Wang Adam Byerly Kate Sanders Adam Byerly Jingyu Zhang Benjamin Van Durme Daniel Khashabi LLMAG 75 6 0 18 Mar 2024
AI Assistance for UX: A Literature Review Through Human-Centered AI Yuwen Lu Yuewen Yang Qinyi Zhao Chengzhi Zhang Toby Jia-Jun Li 24 16 0 08 Feb 2024
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue Xing Han Lù Zdeněk Kasner Siva Reddy 34 60 0 08 Feb 2024
Dual-View Visual Contextualization for Web Navigation Jihyung Kil Chan Hee Song Boyuan Zheng Xiang Deng Yu-Chuan Su Wei-Lun Chao EgoV 22 12 0 06 Feb 2024
Android in the Wild: A Large-Scale Dataset for Android Device Control Christopher Rawles Alice Li Daniel Rodriguez Oriana Riva Timothy Lillicrap LM&Ro 28 143 0 19 Jul 2023
Multimodal Web Navigation with Instruction-Finetuned Foundation Models Hiroki Furuta Kuang-Huei Lee Ofir Nachum Yutaka Matsuo Aleksandra Faust S. Gu Izzeddin Gur LM&Ro 36 93 0 19 May 2023
Language Models can Solve Computer Tasks Geunwoo Kim Pierre Baldi Stephen Marcus McAleer LLMAG LM&Ro 43 342 0 30 Mar 2023
UGIF: UI Grounded Instruction Following S. Venkatesh Partha P. Talukdar S. Narayanan 16 10 0 14 Nov 2022
Enabling Conversational Interaction with Mobile UI using Large Language Models Bryan Wang Gang Li Yang Li 181 132 0 18 Sep 2022
Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization Sungryull Sohn Hyunjae Woo Jongwook Choi lyubing qiang Izzeddin Gur Aleksandra Faust Honglak Lee BDL OffRL 31 3 0 25 May 2022
Environment Generation for Zero-Shot Compositional Reinforcement Learning Izzeddin Gur Natasha Jaques Yingjie Miao Jongwook Choi Manoj Kumar Tiwari Honglak Lee Aleksandra Faust 36 43 0 21 Jan 2022
Learning Context-Aware Representations of Subtrees Cedric Cook 27 2 0 08 Nov 2021
Learning UI Navigation through Demonstrations composed of Macro Actions Wei Li LLMAG 15 9 0 16 Oct 2021
FLIN: A Flexible Natural Language Interface for Web Navigation Sahisnu Mazumder Oriana Riva LRM 48 23 0 24 Oct 2020
Learning Abstract Models for Strategic Exploration and Fast Reward Transfer E. Liu Ramtin Keramati Sudarshan Seshadri Kelvin Guu Panupong Pasupat Emma Brunskill Percy Liang OffRL 27 5 0 12 Jul 2020