Increasing Probability Mass on Answer Choices Does Not Always Improve
Accuracy

Increasing Probability Mass on Answer Choices Does Not Always Improve Accuracy

24 May 2023

Sarah Wiegreffe

Matthew Finlayson

Oyvind Tafjord

Ashish Sabharwal

Papers citing "Increasing Probability Mass on Answer Choices Does Not Always Improve Accuracy"

7 / 7 papers shown

Title
Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions Sarah Wiegreffe Oyvind Tafjord Yonatan Belinkov Hanna Hajishirzi Ashish Sabharwal 50 5 0 21 Jul 2024
OLMES: A Standard for Language Model Evaluations Yuling Gu Oyvind Tafjord Bailey Kuehl Dany Haddad Jesse Dodge Hannaneh Hajishirzi ELM 40 14 0 12 Jun 2024
A Study on Large Language Models' Limitations in Multiple-Choice Question Answering Aisha Khatun Daniel G. Brown ELM 29 13 0 15 Jan 2024
From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning Ming Li Yong Zhang Zhitao Li Jiuhai Chen Lichang Chen Ning Cheng Jianzong Wang Dinesh Manocha Jing Xiao 48 176 0 23 Aug 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 363 12,003 0 04 Mar 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 225 339 0 02 Feb 2022
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 279 1,124 0 18 Apr 2021