ZeroSumEval: An Extensible Framework For Scaling LLM Evaluation with Inter-Model Competition

10 March 2025

Papers citing "ZeroSumEval: An Extensible Framework For Scaling LLM Evaluation with Inter-Model Competition"

20 / 20 papers shown

Title
Measuring General Intelligence with Generated Games Vivek Verma David Huang William Chen Dan Klein Nicholas Tomlin ReLM ELM LM&MA LRM 104 2 0 12 May 2025
Are Large Vision Language Models Good Game Players? Xinyu Wang Bohan Zhuang Qi Wu MLLM ELM LRM 128 5 0 04 Mar 2025
Agent-as-a-Judge: Evaluate Agents with Agents Mingchen Zhuge Changsheng Zhao Dylan R. Ashley Wenyi Wang Dmitrii Khizbullin ... Raghuraman Krishnamoorthi Yuandong Tian Yangyang Shi Vikas Chandra Jürgen Schmidhuber ELM 113 41 0 14 Oct 2024
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations Md Tahmid Rahman Laskar Sawsan Alqahtani M Saiful Bari Mizanur Rahman Mohammad Abdullah Matin Khan ... Chee Wei Tan Md. Rizwan Parvez Enamul Hoque Shafiq Joty Jimmy Huang ELM ALM 81 36 0 04 Jul 2024
CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents Tianqi Xu Linyao Chen Dai-Jie Wu Yanjun Chen Zecheng Zhang ... Shilong Liu Bochen Qian Philip Torr Guohao Li Ge Li 96 20 0 01 Jul 2024
GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents Anthony Costarelli Mat Allen Roman Hauksson Grace Sodunke Suhas Hariharan Carlson Cheng Wenjie Li Joshua Clymer Arjun Yadav ELM ReLM LLMAG LRM 79 24 0 07 Jun 2024
Disentangling Length from Quality in Direct Preference Optimization Ryan Park Rafael Rafailov Stefano Ermon Chelsea Finn ALM 86 139 0 28 Mar 2024
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference Wei-Lin Chiang Lianmin Zheng Ying Sheng Anastasios Nikolas Angelopoulos Tianle Li ... Hao Zhang Banghua Zhu Michael I. Jordan Joseph E. Gonzalez Ion Stoica OSLM 144 574 0 07 Mar 2024
GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations Jinhao Duan Renming Zhang James Diffenderfer B. Kailkhura Lichao Sun Elias Stengel-Eskin Mohit Bansal Tianlong Chen Kaidi Xu ELM LRM 69 64 0 19 Feb 2024
When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards Norah A. Alzahrani H. A. Alyahya Sultan Yazeed Alnumay Muhtasim Tahmid Shaykhah Alsubaie ... Saleh Soltan Nathan Scales Marie-Anne Lachaux Samuel R. Bowman Haidar Khan ELM 116 79 0 01 Feb 2024
NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes Lizhou Fan Wenyue Hua Lingyao Li Haoyang Ling Yongfeng Zhang LRM 82 54 0 22 Dec 2023
DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines Arnav Singhvi Manish Shetty Shangyin Tan Christopher Potts Koushik Sen Matei A. Zaharia Omar Khattab 44 20 0 20 Dec 2023
Elo Uncovered: Robustness and Best Practices in Language Model Evaluation M. Boubdir Edward Kim Beyza Ermis Sara Hooker Marzieh Fadaee ELM 60 40 0 29 Nov 2023
Rethinking Benchmark and Contamination for Language Models with Rephrased Samples Shuo Yang Wei-Lin Chiang Lianmin Zheng Joseph E. Gonzalez Ion Stoica ALM 53 126 0 08 Nov 2023
FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation Tu Vu Mohit Iyyer Xuezhi Wang Noah Constant Jerry W. Wei ... Chris Tar Yun-hsuan Sung Denny Zhou Quoc Le Thang Luong KELM HILM LRM 97 216 0 05 Oct 2023
SmartPlay: A Benchmark for LLMs as Intelligent Agents Yue Wu Xuan Tang Tom Michael Mitchell Yuanzhi Li ELM LLMAG 67 72 0 02 Oct 2023
WebArena: A Realistic Web Environment for Building Autonomous Agents Shuyan Zhou Frank F. Xu Hao Zhu Xuhui Zhou Robert Lo ... Tianyue Ou Yonatan Bisk Daniel Fried Uri Alon Graham Neubig LLMAG 157 463 0 25 Jul 2023
From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought L. Wong Gabriel Grand Alexander K. Lew Noah D. Goodman Vikash K. Mansinghka Jacob Andreas J. Tenenbaum LRM AI4CE 50 106 0 22 Jun 2023
Generative Agents: Interactive Simulacra of Human Behavior J. Park Joseph C. O'Brien Carrie J. Cai Meredith Ringel Morris Percy Liang Michael S. Bernstein LM&Ro AI4CE 392 1,922 0 07 Apr 2023
Dynabench: Rethinking Benchmarking in NLP Douwe Kiela Max Bartolo Yixin Nie Divyansh Kaushik Atticus Geiger ... Pontus Stenetorp Robin Jia Joey Tianyi Zhou Christopher Potts Adina Williams 198 407 0 07 Apr 2021