Goldilocks: Consistent Crowdsourced Scalar Annotations with Relative Uncertainty

4 August 2021

Papers citing "Goldilocks: Consistent Crowdsourced Scalar Annotations with Relative Uncertainty"

11 / 11 papers shown

Title
Negotiative Alignment: Embracing Disagreement to Achieve Fairer Outcomes -- Insights from Urban Studies Rashid Mushkani Hugo Berard Shin Koseki 186 1 0 16 Mar 2025
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty Umang Bhatt Javier Antorán Yunfeng Zhang Q. V. Liao P. Sattigeri ... L. Nachman R. Chunara Madhulika Srikumar Adrian Weller Alice Xiang 62 248 0 15 Nov 2020
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models Samuel Gehman Suchin Gururangan Maarten Sap Yejin Choi Noah A. Smith 133 1,191 0 24 Sep 2020
Beyond User Self-Reported Likert Scale Ratings: A Comparison Model for Automatic Dialog Evaluation Weixin Liang James Zou Zhou Yu ELM 57 34 0 21 May 2020
Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods Eyke Hüllermeier Willem Waegeman PER UD 202 1,405 0 21 Oct 2019
Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Mor Geva Yoav Goldberg Jonathan Berant 320 325 0 21 Aug 2019
Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation Samuel Läubli Rico Sennrich M. Volk 37 258 0 21 Aug 2018
Know What You Don't Know: Unanswerable Questions for SQuAD Pranav Rajpurkar Robin Jia Percy Liang RALM ELM 247 2,837 0 11 Jun 2018
Efficient Online Scalar Annotation with Bounded Support Keisuke Sakaguchi Benjamin Van Durme 46 45 0 04 Jun 2018
Quantifying Facial Age by Posterior of Age Comparisons Yunxuan Zhang Li Liu Cheng Li Chen Change Loy CVBM 37 77 0 31 Aug 2017
Crowdsourcing Ground Truth for Medical Relation Extraction Anca Dumitrache Lora Aroyo Chris Welty 46 65 0 09 Jan 2017