GAIA: a benchmark for General AI Assistants

21 November 2023

Grégoire Mialon

Papers citing "GAIA: a benchmark for General AI Assistants"

30 / 30 papers shown

Title
SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesis Shuang Sun Huatong Song Yuhao Wang Ruiyang Ren Jinhao Jiang ... Wayne Xin Zhao Zheng Liu Lei Fang Zhongyuan Wang Ji-Rong Wen LRM 63 5 0 22 May 2025
MCP-RADAR: A Multi-Dimensional Benchmark for Evaluating Tool Use Capabilities in Large Language Models Xuanqi Gao Siyi Xie Juan Zhai Shqing Ma Chao Shen ELM 83 0 0 22 May 2025
lmgame-Bench: How Good are LLMs at Playing Games? Lanxiang Hu Mingjia Huo Yu Zhang Haoyang Yu Eric P. Xing Ion Stoica Tajana Rosing Haojian Jin Hao Zhang 118 1 0 21 May 2025
Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems Shaokun Zhang Ming Yin Jieyu Zhang Jing Liu Zhiguang Han ... Beibin Li Chi Wang Hongru Wang Yuxiao Chen Qingyun Wu 144 7 0 30 Apr 2025
BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese Peilin Zhou Bruce Leon Xiang Ying Chen Zhang Yifan Shao ... Sixin Hong J. Ren Jian Chen Chao-Hong Liu Yining Hua RALM ELM LRM 103 5 0 27 Apr 2025
Synergizing RAG and Reasoning: A Systematic Review Yunfan Gao Yun Xiong Yijie Zhong Yuxi Bi Ming Xue Haoyu Wang LRM AI4CE 410 7 0 22 Apr 2025
MARFT: Multi-Agent Reinforcement Fine-Tuning Junwei Liao Muning Wen Jun Wang Weinan Zhang OffRL 117 5 0 21 Apr 2025
Planet as a Brain: Towards Internet of AgentSites based on AIOS Server Xiang Zhang Yongfeng Zhang 105 0 0 19 Apr 2025
Affordable AI Assistants with Knowledge Graph of Thoughts Maciej Besta Lorenzo Paleari Jia Hao Andrea Jiang Robert Gerstenberger You Wu ... Torsten Hoefler Grzegorz Kwa'sniewski Marcin Copik H. Niewiadomski Torsten Hoefler LLMAG RALM 489 0 0 03 Apr 2025
Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents Shuo Ren Pu Jian Zhenjiang Ren Chunlin Leng Can Xie Jiajun Zhang LLMAG AI4CE 135 4 0 31 Mar 2025
Measuring AI Ability to Complete Long Tasks Thomas Kwa Ben West Joel Becker Amy Deng Katharyn Garcia ... Lucas Jun Koba Sato H. Wijk Daniel M. Ziegler Elizabeth Barnes Lawrence Chan ELM 236 18 0 18 Mar 2025
Why Do Multi-Agent LLM Systems Fail? Mert Cemri Melissa Z. Pan Shuyi Yang Lakshya A Agrawal Bhavya Chopra ... Dan Klein Kannan Ramchandran Matei A. Zaharia Joseph E. Gonzalez Ion Stoica LLMAG Presented at ResearchTrend Connect \| LLMAG on 23 Apr 2025 217 37 0 17 Mar 2025
AgentStudio: A Toolkit for Building General Virtual Agents Longtao Zheng Zhiyuan Huang Zhenghai Xue Xinrun Wang Bo An Shuicheng Yan 200 20 0 17 Feb 2025
Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agents Vardaan Pahuja Yadong Lu Corby Rosset Boyu Gou Arindam Mitra Spencer Whitehead Yu Su Ahmed Awadallah LLMAG LM&Ro Presented at ResearchTrend Connect \| LLMAG on 14 Mar 2025 208 7 1 17 Feb 2025
WebWalker: Benchmarking LLMs in Web Traversal Jialong Wu Wenbiao Yin Yong Jiang Zhenglin Wang Zekun Xi ... Linhai Zhang Yulan He Deyu Zhou Pengjun Xie Fei Huang 108 14 0 13 Jan 2025
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents Christopher Rawles Sarah Clinckemaillie Yifan Chang Jonathan Waltz Gabrielle Lau ... Daniel Toyama Robert Berry Divya Tyamagundlu Timothy Lillicrap Oriana Riva LLMAG 124 72 0 23 May 2024
AIOS: LLM Agent Operating System Kai Mei Zelong Li Wujiang Xu Wenyue Hua Mingyu Jin Yongfeng Zhang Shuyuan Xu Ruosong Ye Yingqiang Ge Yongfeng Zhang LLMAG 86 25 0 25 Mar 2024
TDAG: A Multi-Agent Framework based on Dynamic Task Decomposition and Agent Generation Yaoxiang Wang Zhiyong Wu Junfeng Yao Jinsong Su LLMAG 125 11 0 15 Feb 2024
Gentopia: A Collaborative Platform for Tool-Augmented LLMs Binfeng Xu Xukun Liu Hua Shen Zeyu Han Yuhan Li Murong Yue Zhi-Ping Peng Yuchen Liu Ziyu Yao Dongkuan Xu LLMAG 64 19 0 08 Aug 2023
AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learn Difei Gao Lei Ji Luowei Zhou Kevin Lin Joya Chen Zihan Fan Mike Zheng Shou MLLM 73 76 0 14 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 405 4,422 0 09 Jun 2023
Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents Yashar Talebirad Amirhossein Nadiri LLMAG 112 233 0 05 Jun 2023
On the Tool Manipulation Capability of Open-source Large Language Models Qiantong Xu Fenglu Hong Yangqiu Song Changran Hu Zheng Chen Jian Zhang LLMAG 87 76 0 25 May 2023
Gorilla: Large Language Model Connected with Massive APIs Shishir G. Patil Tianjun Zhang Xin Wang Joseph E. Gonzalez ELM CLL ALM SyDa 86 566 0 24 May 2023
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 418 3,602 0 29 Apr 2022
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language Andy Zeng Maria Attarian Brian Ichter K. Choromanski Adrian S. Wong ... Michael S. Ryoo Vikas Sindhwani Johnny Lee Vincent Vanhoucke Peter R. Florence ReLM LRM 152 588 0 01 Apr 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 326 4,569 0 27 Oct 2021
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 274 2,323 0 02 May 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,196 0 20 Apr 2018
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 312 8,169 0 16 Jun 2016