Changing Answer Order Can Decrease MMLU Accuracy

Changing Answer Order Can Decrease MMLU Accuracy

27 June 2024

Candace Ross

Papers citing "Changing Answer Order Can Decrease MMLU Accuracy"

10 / 10 papers shown

Title
When2Call: When (not) to Call Tools Hayley Ross Ameya Sunil Mahabaleshwarkar Yoshi Suhara 93 0 0 26 Apr 2025
Stress Testing Generalization: How Minor Modifications Undermine Large Language Model Performance Guangxiang Zhao Saier Hu Xiaoqi Jian Jinzhu Wu Yuhan Wu Change Jia Lin Sun Xiangzheng Zhang 93 0 0 18 Feb 2025
On Memorization of Large Language Models in Logical Reasoning Chulin Xie Yangsibo Huang Chiyuan Zhang Da Yu Xinyun Chen Bill Yuchen Lin Bo Li Badih Ghazi Ravi Kumar LRM 53 20 0 30 Oct 2024
Improving Model Evaluation using SMART Filtering of Benchmark Datasets Vipul Gupta Candace Ross David Pantoja R. Passonneau Megan Ung Adina Williams 76 1 0 26 Oct 2024
Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machina Yuan Gao Dokyun Lee Gordon Burtch Sina Fazelpour LRM 56 7 0 25 Oct 2024
CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning Joshua Ong Jun Leang Aryo Pradipta Gema Shay B. Cohen ReLM LRM ReCod 40 2 0 14 Oct 2024
Mitigating Selection Bias with Node Pruning and Auxiliary Options Hyeong Kyu Choi Weijie Xu Chi Xue Stephanie Eckman Chandan K. Reddy 29 1 0 27 Sep 2024
Composable Interventions for Language Models Arinbjorn Kolbeinsson Kyle O'Brien Tianjin Huang Shanghua Gao Shiwei Liu ... Anurag J. Vaidya Faisal Mahmood Marinka Zitnik Tianlong Chen Thomas Hartvigsen KELM MU 89 5 0 09 Jul 2024
The Curious Case of Absolute Position Embeddings Koustuv Sinha Amirhossein Kazemnejad Siva Reddy J. Pineau Dieuwke Hupkes Adina Williams 87 15 0 23 Oct 2022
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,919 0 31 Dec 2020