Title
How Can I Publish My LLM Benchmark Without Giving the True Answers Away? Takashi Ishida Thanawat Lodkaew Ikko Yamane 203 0 0 23 May 2025
The FACTS Grounding Leaderboard: Benchmarking LLMs' Ability to Ground Responses to Long-Form Input Alon Jacovi Andrew Wang Chris Alberti Connie Tao Jon Lipovetz ... Rachana Fellinger Rui Wang Zizhao Zhang Sasha Goldshtein Dipanjan Das HILM ALM 193 17 0 06 Jan 2025
LiveBench: A Challenging, Contamination-Limited LLM Benchmark Colin White Samuel Dooley Manley Roberts Arka Pal Ben Feuer ... Willie Neiswanger Micah Goldblum Tom Goldstein Willie Neiswanger Micah Goldblum ELM 118 20 0 27 Jun 2024
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code Naman Jain King Han Alex Gu Wen-Ding Li Fanjia Yan Tianjun Zhang Sida I. Wang Armando Solar-Lezama Koushik Sen Ion Stoica ELM 148 448 0 12 Mar 2024
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference Wei-Lin Chiang Lianmin Zheng Ying Sheng Anastasios Nikolas Angelopoulos Tianle Li ... Hao Zhang Banghua Zhu Michael I. Jordan Joseph E. Gonzalez Ion Stoica OSLM 160 602 0 07 Mar 2024
Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs Simone Balloccu Patrícia Schmidtová Mateusz Lango Ondrej Dusek SILM ELM PILM 103 181 0 06 Feb 2024
NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark Oscar Sainz Jon Ander Campos Iker García-Ferrero Julen Etxaniz Oier López de Lacalle Eneko Agirre 80 185 0 27 Oct 2023
Proving Test Set Contamination in Black Box Language Models Yonatan Oren Nicole Meister Niladri Chatterji Faisal Ladhak Tatsunori B. Hashimoto HILM 122 146 0 26 Oct 2023
SWE-bench: Can Language Models Resolve Real-World GitHub Issues? Carlos E. Jimenez John Yang Alexander Wettig Shunyu Yao Kexin Pei Ofir Press Karthik Narasimhan ELM 138 647 0 10 Oct 2023
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 399 4,606 0 27 Oct 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 245 5,695 0 07 Jul 2021
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika Basel Alomair Jacob Steinhardt ELM RALM 201 4,580 0 07 Sep 2020
HellaSwag: Can a Machine Really Finish Your Sentence? Rowan Zellers Ari Holtzman Yonatan Bisk Ali Farhadi Yejin Choi 194 2,534 0 19 May 2019
Do ImageNet Classifiers Generalize to ImageNet? Benjamin Recht Rebecca Roelofs Ludwig Schmidt Vaishaal Shankar OOD SSeg VLM 138 1,732 0 13 Feb 2019
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 320 8,189 0 16 Jun 2016
OpenML: networked science in machine learning Joaquin Vanschoren Jan N. van Rijn B. Bischl Luís Torgo FedML AI4CE 193 1,334 0 29 Jul 2014