Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation

22 October 2023

Papers citing "Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation"

34 / 34 papers shown

Title
Language Imbalance Driven Rewarding for Multilingual Self-improving Wen Yang Junhong Wu Chen Wang Chengqing Zong J.N. Zhang ALM LRM 164 7 0 11 Oct 2024
Sample-Efficient Human Evaluation of Large Language Models via Maximum Discrepancy Competition Kehua Feng Keyan Ding Hongzhi Tan Kede Ma Zhihua Wang ... Yuzhou Cheng Ge Sun Guozhou Zheng Qiang Zhang H. Chen 80 12 0 10 Apr 2024
Benchmarking Large Language Model Capabilities for Conditional Generation Joshua Maynez Priyanka Agrawal Sebastian Gehrmann ELM LM&MA 75 31 0 29 Jun 2023
Xiezhi: An Ever-Updating Benchmark for Holistic Domain Knowledge Evaluation Zhouhong Gu Xiaoxuan Zhu Haoning Ye Lin Zhang Jianchen Wang ... Zili Wang Shusen Wang Weiguo Zheng Hongwei Feng Yanghua Xiao ALM ELM 125 60 0 09 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 391 4,388 0 09 Jun 2023
The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only Guilherme Penedo Quentin Malartic Daniel Hesslow Ruxandra-Aimée Cojocaru Alessandro Cappelli Hamza Alobeidli B. Pannier Ebtesam Almazrouei Julien Launay 118 772 0 01 Jun 2023
Koala: An Index for Quantifying Overlaps with Pre-training Corpora Thuy-Trang Vu Xuanli He Gholamreza Haffari Ehsan Shareghi CLL 50 15 0 26 Mar 2023
Fine-tuning language models to find agreement among humans with diverse preferences Michiel A. Bakker Martin Chadwick Hannah R. Sheahan Michael Henry Tessler Lucy Campbell-Gillingham ... Nat McAleese Amelia Glaese John Aslanides M. Botvinick Christopher Summerfield ALM 107 233 0 28 Nov 2022
A Close Look into the Calibration of Pre-trained Language Models Yangyi Chen Lifan Yuan Ganqu Cui Zhiyuan Liu Heng Ji 129 51 0 31 Oct 2022
Out of the BLEU: how should we assess quality of the Code Generation models? Mikhail Evtikhiev Egor Bogomolov Yaroslav Sokolov T. Bryksin ALM 78 95 0 05 Aug 2022
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code Sebastian Gehrmann Abhik Bhattacharjee Abinaya Mahendiran Alex Jinpeng Wang Alexandros Papangelis ... Yacine Jernite Yi Xu Yisi Sang Yixin Liu Yufang Hou 91 38 0 22 Jun 2022
A global analysis of metrics used for measuring performance in natural language processing Kathrin Blagec Georg Dorffner M. Moradi Simon Ott Matthias Samwald 76 28 0 25 Apr 2022
Teaching language models to support answers with verified quotes Jacob Menick Maja Trebacz Vladimir Mikulik John Aslanides Francis Song ... Mia Glaese Susannah Young Lucy Campbell-Gillingham G. Irving Nat McAleese ELM RALM 300 265 0 21 Mar 2022
Towards Responsible Natural Language Annotation for the Varieties of Arabic A. S. Bergman Mona T. Diab 85 19 0 17 Mar 2022
Dynamic Human Evaluation for Relative Model Comparisons Thórhildur Thorleiksdóttir Cédric Renggli Nora Hollenstein Ce Zhang 66 2 0 15 Dec 2021
A General Language Assistant as a Laboratory for Alignment Amanda Askell Yuntao Bai Anna Chen Dawn Drain Deep Ganguli ... Tom B. Brown Jack Clark Sam McCandlish C. Olah Jared Kaplan ALM 118 779 0 01 Dec 2021
Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models Wei Ping Chejian Xu Shuohang Wang Zhe Gan Yu Cheng Jianfeng Gao Ahmed Hassan Awadallah Yangqiu Song VLM ELM AAML 65 225 0 04 Nov 2021
LMdiff: A Visual Diff Tool to Compare Language Models Hendrik Strobelt Benjamin Hoover Arvind Satyanarayan Sebastian Gehrmann VLM 66 19 0 02 Nov 2021
Better than Average: Paired Evaluation of NLP Systems Maxime Peyrard Wei Zhao Steffen Eger Robert West ELM 73 26 0 20 Oct 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 348 1,706 0 15 Oct 2021
SaFeRDialogues: Taking Feedback Gracefully after Conversational Safety Failures Megan Ung Jing Xu Y-Lan Boureau 68 47 0 14 Oct 2021
BARTScore: Evaluating Generated Text as Text Generation Weizhe Yuan Graham Neubig Pengfei Liu 109 843 0 22 Jun 2021
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus Jesse Dodge Maarten Sap Ana Marasović William Agnew Gabriel Ilharco Dirk Groeneveld Margaret Mitchell Matt Gardner AILaw 118 448 0 18 Apr 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics Sebastian Gehrmann Tosin Adewumi Karmanya Aggarwal Pawan Sasanka Ammanamanchi Aremu Anuoluwapo ... Nishant Subramani Wei Xu Diyi Yang Akhila Yerukola Jiawei Zhou VLM 311 285 0 02 Feb 2021
Evaluation of Text Generation: A Survey Asli Celikyilmaz Elizabeth Clark Jianfeng Gao ELM LM&MA 112 387 0 26 Jun 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 447 20,298 0 23 Oct 2019
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 469 1,762 0 18 Sep 2019
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance Wei Zhao Maxime Peyrard Fei Liu Yang Gao Christian M. Meyer Steffen Eger 181 602 0 05 Sep 2019
A Study of BFLOAT16 for Deep Learning Training Dhiraj D. Kalamkar Dheevatsa Mudigere Naveen Mellempudi Dipankar Das K. Banerjee ... Sudarshan Srinivasan Abhisek Kundu M. Smelyanskiy Bharat Kaul Pradeep Dubey MQ 83 347 0 29 May 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 329 5,845 0 21 Apr 2019
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge Alon Talmor Jonathan Herzig Nicholas Lourie Jonathan Berant RALM 140 1,744 0 02 Nov 2018
RankME: Reliable Human Ratings for Natural Language Generation Jekaterina Novikova Ondrej Dusek Verena Rieser ALM 48 109 0 15 Mar 2018
The NarrativeQA Reading Comprehension Challenge Tomás Kociský Jonathan Richard Schwarz Phil Blunsom Chris Dyer Karl Moritz Hermann Gábor Melis Edward Grefenstette 139 779 0 19 Dec 2017
International Standard for a Linguistic Annotation Framework Laurent Romary Nancy Ide 179 284 0 22 Jul 2007