Don't Label Twice: Quantity Beats Quality when Comparing Binary Classifiers on a Budget

3 February 2024

Papers citing "Don't Label Twice: Quantity Beats Quality when Comparing Binary Classifiers on a Budget"

19 / 19 papers shown

Title
Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the data Florian E. Dorner Vivian Y. Nastl Moritz Hardt ELM ALM 67 8 0 17 Oct 2024
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 213 11,636 0 18 Jul 2023
OpenAssistant Conversations -- Democratizing Large Language Model Alignment Andreas Kopf Yannic Kilcher Dimitri von Rutte Sotiris Anagnostidis Zhi Rui Tam ... Arnav Dantuluri Andrew Maguire Christoph Schuhmann Huu Nguyen A. Mattick ALM LM&MA 85 611 0 14 Apr 2023
Human-Guided Fair Classification for Natural Language Processing Florian E.Dorner Momchil Peychev Nikola Konstantinov Naman Goel Elliott Ash Martin Vechev FaML 54 4 0 20 Dec 2022
To Aggregate or Not? Learning with Separate Noisy Labels Jiaheng Wei Zhaowei Zhu Tianyi Luo Ehsan Amid Abhishek Kumar Yang Liu NoLa 48 38 0 14 Jun 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 717 12,525 0 04 Mar 2022
Jury Learning: Integrating Dissenting Voices into Machine Learning Models Mitchell L. Gordon Michelle S. Lam J. Park Kayur Patel Jeffrey T. Hancock Tatsunori Hashimoto Michael S. Bernstein 63 149 0 07 Feb 2022
Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation Emily L. Denton Mark Díaz Ian D Kivlichan Vinodkumar Prabhakaran Rachel Rosen 47 67 0 08 Dec 2021
Clean or Annotate: How to Spend a Limited Data Collection Budget Derek Chen Zhou Yu Samuel R. Bowman 55 13 0 15 Oct 2021
Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations Aida Mostafazadeh Davani Mark Díaz Vinodkumar Prabhakaran 35 312 0 12 Oct 2021
On Releasing Annotator-Level Labels and Information in Datasets Vinodkumar Prabhakaran Aida Mostafazadeh Davani Mark Díaz 64 147 0 12 Oct 2021
Why do classifier accuracies show linear trends under distribution shift? Horia Mania S. Sra OOD 57 19 0 31 Dec 2020
Do ImageNet Classifiers Generalize to ImageNet? Benjamin Recht Rebecca Roelofs Ludwig Schmidt Vaishaal Shankar OOD SSeg VLM 90 1,693 0 13 Feb 2019
Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion Ryutaro Tanno A. Saeedi S. Sankaranarayanan Daniel C. Alexander N. Silberman NoLa 54 230 0 10 Feb 2019
Crowd disagreement about medical images is informative Veronika Cheplygina J. Pluim 39 24 0 21 Jun 2018
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 439 4,444 0 18 Apr 2017
A large annotated corpus for learning natural language inference Samuel R. Bowman Gabor Angeli Christopher Potts Christopher D. Manning 245 4,268 0 21 Aug 2015
The Ladder: A Reliable Leaderboard for Machine Learning Competitions Avrim Blum Moritz Hardt 54 134 0 16 Feb 2015
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.2K 39,383 0 01 Sep 2014