The Perspectivist Paradigm Shift: Assumptions and Challenges of
Capturing Human Labels

The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels

9 May 2024

Su Lin Blodgett

Papers citing "The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels"

11 / 11 papers shown

Title
Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions Matthias Orlikowski Jiaxin Pei Paul Röttger Philipp Cimiano David Jurgens Dirk Hovy 59 1 0 28 Feb 2025
Correcting Annotator Bias in Training Data: Population-Aligned Instance Replication (PAIR) Stephanie Eckman Bolei Ma Christoph Kern Rob Chew Barbara Plank Frauke Kreuter 41 0 0 12 Jan 2025
Beyond Dataset Creation: Critical View of Annotation Variation and Bias Probing of a Dataset for Online Radical Content Detection Arij Riabi Virginie Mouilleron Menel Mahamdi Wissam Antoun Djamé Seddah 72 0 0 16 Dec 2024
Sensitive Content Classification in Social Media: A Holistic Resource and Evaluation Dimosthenis Antypas Indira Sen Carla Pérez-Almendros Jose Camacho-Collados Francesco Barbieri 69 1 0 29 Nov 2024
Venire: A Machine Learning-Guided Panel Review System for Community Content Moderation Vinay Koshy Frederick Choi Yi-Shyuan Chiang Hari Sundaram Eshwar Chandrasekharan Karrie Karahalios 33 2 0 30 Oct 2024
Which Demographics do LLMs Default to During Annotation? Johannes Schäfer Aidan Combs Christopher Bagdon Jiahui Li Nadine Probol ... Yarik Menchaca Resendiz Aswathy Velutharambath Amelie Wuhrl Sabine Weber Roman Klinger 35 2 0 11 Oct 2024
Incorporating Worker Perspectives into MTurk Annotation Practices for NLP Olivia Huang Eve Fleisig Dan Klein 38 15 0 06 Nov 2023
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice Fernando Delgado Stephen Yang Michael A. Madaio Qian Yang 76 100 0 02 Oct 2023
Stop Measuring Calibration When Humans Disagree Joris Baan Wilker Aziz Barbara Plank Raquel Fernández 24 53 0 28 Oct 2022
Disembodied Machine Learning: On the Illusion of Objectivity in NLP Zeerak Talat Smarika Lulz Joachim Bingel Isabelle Augenstein 96 51 0 28 Jan 2021
Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Mor Geva Yoav Goldberg Jonathan Berant 242 320 0 21 Aug 2019