Elo Uncovered: Robustness and Best Practices in Language Model
Evaluation

Elo Uncovered: Robustness and Best Practices in Language Model Evaluation

29 November 2023

Papers citing "Elo Uncovered: Robustness and Best Practices in Language Model Evaluation"

13 / 13 papers shown

Title
am-ELO: A Stable Framework for Arena-based LLM Evaluation Zirui Liu Jiatong Li Yan Zhuang Qiang Liu Shuanghong Shen Jie Ouyang Mingyue Cheng Shijin Wang 41 0 0 06 May 2025
ZeroSumEval: An Extensible Framework For Scaling LLM Evaluation with Inter-Model Competition H. A. Alyahya Haidar Khan Yazeed Alnumay M Saiful Bari B. Yener LRM 65 1 0 10 Mar 2025
Varco Arena: A Tournament Approach to Reference-Free Benchmarking Large Language Models Seonil Son Ju-Min Oh Heegon Jin Cheolhun Jang Jeongbeom Jeong Kuntae Kim 44 0 0 20 Feb 2025
Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gap Hyunwoo Ko Guijin Son Dasol Choi RALM LRM 78 7 0 05 Jan 2025
Soft Condorcet Optimization for Ranking of General Agents Marc Lanctot Kate Larson Michael Kaisers Quentin Berthet I. Gemp Manfred Diaz Roberto-Rafael Maura-Rivero Yoram Bachrach Anna Koop Doina Precup 44 0 0 31 Oct 2024
CREAM: Comparison-Based Reference-Free ELO-Ranked Automatic Evaluation for Meeting Summarization Ziwei Gong Lin Ai Harshsaiprasad Deshpande Alexander Johnson Emmy Phung Zehui Wu Ahmad Emami Julia Hirschberg 41 2 0 17 Sep 2024
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations Md Tahmid Rahman Laskar Sawsan Alqahtani M Saiful Bari Mizanur Rahman Mohammad Abdullah Matin Khan ... Chee Wei Tan Md. Rizwan Parvez Enamul Hoque Shafiq R. Joty Jimmy Huang ELM ALM 29 28 0 04 Jul 2024
PRISM: A Design Framework for Open-Source Foundation Model Safety Terrence Neumann Bryan Jones 42 1 0 14 Jun 2024
Prediction-Powered Ranking of Large Language Models Ivi Chatzi Eleni Straitouri Suhas Thejaswi Manuel Gomez Rodriguez ALM 29 5 0 27 Feb 2024
SportsMetrics: Blending Text and Numerical Data to Understand Information Fusion in LLMs Yebowen Hu Kaiqiang Song Sangwoo Cho Xiaoyang Wang H. Foroosh Dong Yu Fei Liu 23 8 0 15 Feb 2024
Evaluating Language Model Agency through Negotiations Tim R. Davidson V. Veselovsky Martin Josifoski Maxime Peyrard Antoine Bosselut Michal Kosinski Robert West LLMAG 34 22 0 09 Jan 2024
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021
Elo Ratings for Large Tournaments of Software Agents in Asymmetric Games Ben P. Wise 11 2 0 23 Apr 2021