ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks

27 March 2023

Papers citing "ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks"

50 / 493 papers shown

Title
CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraints Anirudh Atmakuru Jatin Nainani Rohith Siddhartha Reddy Bheemreddy Anirudh Lakkaraju Zonghai Yao Hamed Zamani Haw-Shiuan Chang 116 2 0 05 Oct 2024
Misinformation with Legal Consequences (MisLC): A New Task Towards Harnessing Societal Harm of Misinformation Chu Fei Luo Radin Shayanfar R. Bhambhoria Samuel Dahan Xiaodan Zhu AILaw 31 0 0 04 Oct 2024
Are Expert-Level Language Models Expert-Level Annotators? Yu-Min Tseng Wei-Lin Chen Chung-Chi Chen Hsin-Hsi Chen ALM 39 1 0 04 Oct 2024
On Unsupervised Prompt Learning for Classification with Black-box Language Models Zhen-Yu Zhang Jiandong Zhang Huaxiu Yao Gang Niu Masashi Sugiyama 26 2 0 04 Oct 2024
Hate Personified: Investigating the role of LLMs in content moderation Sarah Masud Sahajpreet Singh Viktor Hangya Alexander Fraser Tanmoy Chakraborty 30 7 0 03 Oct 2024
Mixed-Session Conversation with Egocentric Memory Jihyoung Jang Taeyoung Kim Hyounghun Kim 33 0 0 03 Oct 2024
Comparing Criteria Development Across Domain Experts, Lay Users, and Models in Large Language Model Evaluation Annalisa Szymanski Simret Araya Gebreegziabher Oghenemaro Anuyah Ronald A Metoyer T. Li ALM ELM 40 7 0 02 Oct 2024
Evaluating Robustness of Reward Models for Mathematical Reasoning Sunghwan Kim Dongjin Kang Taeyoon Kwon Hyungjoo Chae Jungsoo Won Dongha Lee Jinyoung Yeo 38 5 0 02 Oct 2024
'Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants Shivani Kapania William Agnew Motahhare Eslami Hoda Heidari Sarah E Fox 42 4 0 28 Sep 2024
Learning to Love Edge Cases in Formative Math Assessment: Using the AMMORE Dataset and Chain-of-Thought Prompting to Improve Grading Accuracy Owen Henkel Hannah Horne-Robinson Maria Dyshel Nabil Ch Baptiste Moreau-Pernet Ralph Abood 37 0 0 26 Sep 2024
Setting the AI Agenda -- Evidence from Sweden in the ChatGPT Era Bastiaan Bruinsma Annika Fredén Kajsa Hansson Moa Johansson Pasko Kisić-Merino Denitsa Saynova 31 0 0 25 Sep 2024
AI Can Be Cognitively Biased: An Exploratory Study on Threshold Priming in LLM-Based Batch Relevance Assessment Nuo Chen Jiqun Liu Xiaoyu Dong Qijiong Liu Tetsuya Sakai Xiao-Ming Wu 32 10 0 24 Sep 2024
Learning to Localize Actions in Instructional Videos with LLM-Based Multi-Pathway Text-Video Alignment Yuxiao Chen Keqin Li Wentao Bao Deep Patel Yu Kong Martin Renqiang Min Dimitris N. Metaxas DiffM 46 1 0 22 Sep 2024
Collaborative Human-AI Risk Annotation: Co-Annotating Online Incivility with CHAIRA J. Park Rahul Dev Ellezhuthil Pamela J. Wisniewski Vivek Singh 37 1 0 21 Sep 2024
MirrorStories: Reflecting Diversity through Personalized Narrative Generation with Large Language Models Sarfaroz Yunusov Hamza Sidat Ali Emami 76 0 0 20 Sep 2024
What Would You Ask When You First Saw $a^2+b^2=c^2$ ? Evaluating LLM on Curiosity-Driven Questioning Shashidhar Reddy Javaji Zining Zhu ELM ALM 39 0 0 19 Sep 2024
LLM-Measure: Generating Valid, Consistent, and Reproducible Text-Based Measures for Social Science Research Yi Yang Hanyu Duan Jiaxin Liu Kar Yan Tam 23 0 0 19 Sep 2024
Human Interest or Conflict? Leveraging LLMs for Automated Framing Analysis in TV Shows David Alonso del Barrio Max Tiel D. Gática-Pérez 41 3 0 19 Sep 2024
ARTICLE: Annotator Reliability Through In-Context Learning Sujan Dutta Deepak Pandita Tharindu Cyril Weerasooriya Marcos Zampieri Christopher M. Homan Ashiqur R. KhudaBukhsh 32 0 0 18 Sep 2024
Learning variant product relationship and variation attributes from e-commerce website structures Pedro Herrero-Vidal You-Lin Chen Cris Liu Prithviraj Sen Lichao Wang 36 0 0 17 Sep 2024
Model-in-the-Loop (MILO): Accelerating Multimodal AI Data Annotation with LLMs Yifan Wang David Stevens Pranay Shah Wenwen Jiang Miao Liu ... Boying Gong Daniel Lee Jiabo Hu Ning Zhang Bob Kamma 40 1 0 16 Sep 2024
Benchmarking LLMs in Political Content Text-Annotation: Proof-of-Concept with Toxicity and Incivility Data Bastián González-Bustamante 20 2 0 15 Sep 2024
Enhancing Text Annotation through Rationale-Driven Collaborative Few-Shot Prompting Jianfei Wu Xubin Wang Weijia Jia 30 1 0 15 Sep 2024
Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AI Nicholas Pangakis Samuel Wolken 31 3 0 14 Sep 2024
Safeguarding Decentralized Social Media: LLM Agents for Automating Community Rule Compliance Lucio La Cava Andrea Tagarelli LLMAG 30 0 0 13 Sep 2024
Your Weak LLM is Secretly a Strong Teacher for Alignment Leitian Tao Yixuan Li 88 5 0 13 Sep 2024
HexaCoder: Secure Code Generation via Oracle-Guided Synthetic Training Data Hossein Hajipour Lea Schönherr Thorsten Holz Mario Fritz AAML SyDa 26 0 0 10 Sep 2024
Column Vocabulary Association (CVA): semantic interpretation of dataless tables Margherita Martorana Xueli Pan Benno Kruit Tobias Kuhn Jacco van Ossenbruggen 33 1 0 06 Sep 2024
Content Moderation by LLM: From Accuracy to Legitimacy Tao Huang AILaw 37 3 0 05 Sep 2024
Political DEBATE: Efficient Zero-shot and Few-shot Classifiers for Political Text Michael Burnham Kayla Kahn Ryan Yank Wang Rachel X. Peng 40 5 0 03 Sep 2024
It is Time to Develop an Auditing Framework to Promote Value Aware Chatbots Yanchen Wang Lisa Singh 31 1 0 03 Sep 2024
Towards Empathetic Conversational Recommender Systems Xiaoyu Zhang Ruobing Xie Yougang Lyu Xin Xin Pengjie Ren Mingfei Liang Bo Zhang Zhanhui Kang Maarten de Rijke Zhaochun Ren 38 6 0 30 Aug 2024
From Text to Emotion: Unveiling the Emotion Annotation Capabilities of LLMs Minxue Niu Mimansa Jaiswal Emily Mower Provost 38 5 0 30 Aug 2024
Can Unconfident LLM Annotations Be Used for Confident Conclusions? Kristina Gligorić Tijana Zrnic Cinoo Lee Emmanuel J. Candès Dan Jurafsky 72 6 0 27 Aug 2024
Utilizing Large Language Models for Named Entity Recognition in Traditional Chinese Medicine against COVID-19 Literature: Comparative Study Xu Tong N. Smirnova Sharmila Upadhyaya Ran Yu Jack H. Culbert Chao Sun Wolfgang Otto Philipp Mayr AI4MH 29 0 0 24 Aug 2024
Intelligent OPC Engineer Assistant for Semiconductor Manufacturing Guojin Chen Haoyu Yang Bei Yu Haoxing Ren 44 0 0 23 Aug 2024
Critique-out-Loud Reward Models Zachary Ankner Mansheej Paul Brandon Cui Jonathan D. Chang Prithviraj Ammanabrolu ALM LRM 43 30 0 21 Aug 2024
How Susceptible are LLMs to Influence in Prompts? Sotiris Anagnostidis Jannis Bulian LRM 40 16 0 17 Aug 2024
SEAL: Systematic Error Analysis for Value ALignment Manon Revel Matteo Cargnelutti Tyna Eloundou Greg Leppert 40 3 0 16 Aug 2024
LLaVA-Surg: Towards Multimodal Surgical Assistant via Structured Surgical Video Learning Jiajie Li Garrett C Skinner Gene Yang Brian R Quaranto Steven D. Schwaitzberg Peter C W Kim Jinjun Xiong 38 10 0 15 Aug 2024
What Color Scheme is More Effective in Assisting Readers to Locate Information in a Color-Coded Article? Ho Yin Ng Zeyu He Ting-Hao 'Kenneth' Huang 28 0 0 12 Aug 2024
Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models Shachi H. Kumar Saurav Sahay Sahisnu Mazumder Eda Okur R. Manuvinakurike Nicole Beckage Hsuan Su Hung-yi Lee L. Nachman ELM 42 16 0 07 Aug 2024
Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Construction Albert Sawczyn Katsiaryna Viarenich Konrad Wojtasik Aleksandra Domogała Marcin Oleksy Maciej Piasecki Tomasz Kajdanowicz 42 0 0 05 Aug 2024
The Implications of Open Generative Models in Human-Centered Data Science Work: A Case Study with Fact-Checking Organizations Robert Wolfe Tanushree Mitra 46 2 0 04 Aug 2024
Automated Review Generation Method Based on Large Language Models Shican Wu Xiao Ma Dehui Luo Lulu Li Xiangcheng Shi ... Ran Luo Chunlei Pei Zhijian Zhao Zhi-Jian Zhao Jinlong Gong 77 0 0 30 Jul 2024
VolDoGer: LLM-assisted Datasets for Domain Generalization in Vision-Language Tasks Juhwan Choi Junehyoung Kwon Jungmin Yun Seunguk Yu Youngbin Kim 46 1 0 29 Jul 2024
Logic Distillation: Learning from Code Function by Function for Planning and Decision-making Dong Chen Shilin Zhang Fei Gao Yueting Zhuang Siliang Tang Qidong Liu Mingliang Xu LRM 33 0 0 28 Jul 2024
Towards a Multidimensional Evaluation Framework for Empathetic Conversational Systems Aravind Sesagiri Raamkumar Siyuan Brandon Loh 43 0 0 26 Jul 2024
LLMs left, right, and center: Assessing GPT's capabilities to label political bias from web domains Raphael Hernandes 26 4 0 19 Jul 2024
Dynamic Sentiment Analysis with Local Large Language Models using Majority Voting: A Study on Factors Affecting Restaurant Evaluation Junichiro Niimi 35 3 0 18 Jul 2024