VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks

24 January 2024

Graham Neubig

Daniel Fried

Papers citing "VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks"

41 / 41 papers shown

Title
LiteCUA: Computer as MCP Server for Computer-Use Agent on AIOS Kai Mei Xi Zhu Hang Gao Shuhang Lin Yongfeng Zhang 88 0 0 24 May 2025
A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron? Ada Chen Yongjiang Wu Jing Zhang Shu Yang Jen-tse Huang Kun Wang Wenxuan Wang Wenxuan Wang S. Wang ELM 43 1 0 16 May 2025
WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks Ivan Evtimov Arman Zharmagambetov Aaron Grattafiori Chuan Guo Kamalika Chaudhuri AAML 49 2 0 22 Apr 2025
An Illusion of Progress? Assessing the Current State of Web Agents Tianci Xue Weijian Qi Tianneng Shi Chan Hee Song Boyu Gou D. Song Huan Sun Yu Su LLMAG ELM Presented at ResearchTrend Connect \| LLMAG on 21 May 2025 166 8 1 02 Apr 2025
A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Models Liangbo Ning Ziran Liang Zhuohang Jiang Haohao Qu Yujuan Ding ... Xiao Wei Shanru Lin Hui Liu Philip S. Yu Qing Li LLMAG LM&Ro 115 7 0 30 Mar 2025
UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction Shravan Nayak Xiangru Jian Kevin Qinghong Lin Juan A. Rodriguez Montek Kalsi ... David Vazquez Christopher Pal Perouz Taslakian Spandana Gella Sai Rajeswar 408 1 0 19 Mar 2025
AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents Arman Zharmagambetov Chuan Guo Ivan Evtimov Maya Pavlova Ruslan Salakhutdinov Kamalika Chaudhuri LLMAG 81 4 0 12 Mar 2025
Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks Lutfi Eren Erdogan Nicholas Lee Sehoon Kim Suhong Moon Hiroki Furuta Gopala Anumanchipalli Kemal Kurniawan Amir Gholami LLMAG LM&Ro AIFin 128 2 0 12 Mar 2025
A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrieval Yu Zhang Shutong Qiao Jiaqi Zhang Tzu-Heng Lin Chen Gao Yongqian Li LM&Ro LM&MA 155 3 0 07 Mar 2025
AgentStudio: A Toolkit for Building General Virtual Agents Longtao Zheng Zhiyuan Huang Zhenghai Xue Xinrun Wang Bo An Shuicheng Yan 112 15 0 17 Feb 2025
InSTA: Towards Internet-Scale Training For Agents Brandon Trabucco Gunnar Sigurdsson Robinson Piramuthu Ruslan Salakhutdinov ALM 123 2 0 10 Feb 2025
PixelWorld: Towards Perceiving Everything as Pixels Zhiheng Lyu Xueguang Ma Wenhu Chen 180 1 0 31 Jan 2025
Multi-Agent Geospatial Copilots for Remote Sensing Workflows Chaehong Lee Varatheepan Paramanayakam Andreas Karatzas Yanan Jian Michael Fore Heming Liao Fuxun Yu Ruopu Li Iraklis Anagnostopoulos Dimitrios Stamoulis 56 3 0 28 Jan 2025
WebWalker: Benchmarking LLMs in Web Traversal Jialong Wu Wenbiao Yin Yong Jiang Zhenglin Wang Zekun Xi ... Linhai Zhang Yulan He Deyu Zhou Pengjun Xie Fei Huang 76 12 0 13 Jan 2025
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks Frank F. Xu Yufan Song Boxuan Li Yuxuan Tang Kritanjali Jain ... Wayne Chi Lawrence Jang Yiqing Xie Shuyan Zhou Graham Neubig LLMAG 157 29 0 18 Dec 2024
The BrowserGym Ecosystem for Web Agent Research Thibault Le Sellier De Chezelles Maxime Gasse Alexandre Lacoste Alexandre Drouin Massimo Caccia ... Siva Reddy Quentin Cappart Graham Neubig Ruslan Salakhutdinov Nicolas Chapados LLMAG 131 12 0 06 Dec 2024
Attacking Vision-Language Computer Agents via Pop-ups Yanzhe Zhang Tao Yu Diyi Yang AAML VLM 60 25 0 04 Nov 2024
VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks Lawrence Jang Yinheng Li Charles Ding Justin Lin Paul Pu Liang Dan Zhao Rogerio Bonatti K. Koishida 68 6 0 24 Oct 2024
Large Language Models Empowered Personalized Web Agents Hongru Cai Yongqi Li Wenjie Wang Fengbin Zhu Xiaoyu Shen Wenjie Li Tat-Seng Chua LLMAG 98 12 0 22 Oct 2024
Beyond Browsing: API-Based Web Agents Yueqi Song Frank F. Xu Shuyan Zhou Graham Neubig 79 19 0 21 Oct 2024
AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents Ke Yang Yao Liu Sapana Chaudhary Rasool Fakoor Pratik Chaudhari George Karypis Huzefa Rangwala LLMAG LM&Ro 83 20 0 17 Oct 2024
COMMA: A Communicative Multimodal Multi-Agent Benchmark Timothy Ossowski Jixuan Chen Danyal Maqbool Zefan Cai Tyler Bradshaw Junjie Hu VLM 64 3 0 10 Oct 2024
ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities Jiarui Lu Thomas Holleis Yizhe Zhang Bernhard Aumayer Feng Nan ... Shen Ma Mengyu Li Guoli Yin Zirui Wang Ruoming Pang LLMAG ELM 71 32 0 08 Aug 2024
ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation Cheng Yang Chufan Shi Yaxin Liu Bo Shui Junjie Wang ... Yuxiang Zhang Gongye Liu Xiaomei Nie Deng Cai Yujiu Yang MLLM LRM 67 25 0 14 Jun 2024
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models Seungone Kim Juyoung Suk Ji Yong Cho Shayne Longpre Chaeeun Kim ... Sean Welleck Graham Neubig Moontae Lee Kyungjae Lee Minjoon Seo ELM ALM LM&MA 130 36 0 09 Jun 2024
M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models Hongyu Wang Jiayu Xu Senwei Xie Ruiping Wang Jialin Li Zhaojie Xie Bin Zhang Chuyan Xiong Xilin Chen ELM VLM LRM 110 6 0 24 May 2024
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents Christopher Rawles Sarah Clinckemaillie Yifan Chang Jonathan Waltz Gabrielle Lau ... Daniel Toyama Robert Berry Divya Tyamagundlu Timothy Lillicrap Oriana Riva LLMAG 85 53 0 23 May 2024
Tur[k]ingBench: A Challenge Benchmark for Web Agents Kevin Xu Yeganeh Kordi Kate Sanders Yizhong Wang Adam Byerly Kate Sanders Adam Byerly Jingyu Zhang Benjamin Van Durme Daniel Khashabi LLMAG 84 6 0 18 Mar 2024
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs Shengbang Tong Zhuang Liu Yuexiang Zhai Yi-An Ma Yann LeCun Saining Xie VLM MLLM 70 311 0 11 Jan 2024
Mixtral of Experts Albert Q. Jiang Alexandre Sablayrolles Antoine Roux A. Mensch Blanche Savary ... Théophile Gervet Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LLMAG 88 1,049 0 08 Jan 2024
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI Xiang Yue Yuansheng Ni Kai Zhang Tianyu Zheng Ruoqi Liu ... Yibo Liu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen OSLM ELM VLM 148 833 0 27 Nov 2023
GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation An Yan Zhengyuan Yang Wanrong Zhu Kevin Qinghong Lin Linjie Li ... Yiwu Zhong Julian McAuley Jianfeng Gao Zicheng Liu Lijuan Wang LLMAG LM&Ro 93 105 0 13 Nov 2023
AgentTuning: Enabling Generalized Agent Abilities for LLMs Aohan Zeng Mingdao Liu Rui Lu Bowen Wang Xiao Liu Yuxiao Dong Jie Tang LM&MA ALM LLMAG 54 172 0 19 Oct 2023
The Rise and Potential of Large Language Model Based Agents: A Survey Zhiheng Xi Wenxiang Chen Xin Guo Wei He Yiwen Ding ... Wenjuan Qin Yongyan Zheng Xipeng Qiu Xuanjing Huan Tao Gui LM&MA LM&Ro 3DV AI4CE 72 907 0 14 Sep 2023
Graph of Thoughts: Solving Elaborate Problems with Large Language Models Maciej Besta Nils Blach Aleš Kubíček Robert Gerstenberger Michal Podstawski ... Joanna Gajda Tomasz Lehmann H. Niewiadomski Piotr Nyczyk Torsten Hoefler LRM AI4CE LM&Ro 85 654 0 18 Aug 2023
WebArena: A Realistic Web Environment for Building Autonomous Agents Shuyan Zhou Frank F. Xu Hao Zhu Xuhui Zhou Robert Lo ... Tianyue Ou Yonatan Bisk Daniel Fried Uri Alon Graham Neubig LLMAG 67 420 0 25 Jul 2023
A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis Izzeddin Gur Hiroki Furuta Austin Huang Mustafa Safdari Yutaka Matsuo Douglas Eck Aleksandra Faust LM&Ro LLMAG 83 210 0 24 Jul 2023
ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots Yu-Chung Hsiao Fedir Zubach Maria Wang Jindong Chen Victor Carbune Jason Lin Maria Wang Yun Zhu Jindong Chen RALM 170 27 0 16 Sep 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 283 3,458 0 29 Apr 2022
Multimodal Few-Shot Learning with Frozen Language Models Maria Tsimpoukelli Jacob Menick Serkan Cabi S. M. Ali Eslami Oriol Vinyals Felix Hill MLLM 125 766 0 25 Jun 2021
Long Range Arena: A Benchmark for Efficient Transformers Yi Tay Mostafa Dehghani Samira Abnar Songlin Yang Dara Bahri Philip Pham J. Rao Liu Yang Sebastian Ruder Donald Metzler 108 706 0 08 Nov 2020