ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks

27 March 2023

Papers citing "ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks"

50 / 490 papers shown

Title
GPTEval: A Survey on Assessments of ChatGPT and GPT-4 Rui Mao Guanyi Chen Xulang Zhang Frank Guerin Erik Cambria ELM LM&MA 30 100 0 24 Aug 2023
SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understanding Tianyu Yu Chengyue Jiang Chao Lou Shen Huang Xiaobin Wang ... Haitao Zheng Ningyu Zhang Pengjun Xie Fei Huang Yong-jia Jiang LRM 55 17 0 21 Aug 2023
Characterizing Information Seeking Events in Health-Related Social Discourse Omar Sharif Madhusudan Basak Tanzia Parvin Ava Scharfstein Alphonso Bradham J. Borodovsky S. Lord S. Preum 21 7 0 17 Aug 2023
Discrete Prompt Compression with Reinforcement Learning Hoyoun Jung Kyung-Joong Kim 19 24 0 17 Aug 2023
DiVa: An Iterative Framework to Harvest More Diverse and Valid Labels from User Comments for Music Hongru Liang Jingyao Liu Yuanxin Xiang Jiachen Du Lanjun Zhou Shushen Pan Wenqiang Lei Sichuan University 23 0 0 09 Aug 2023
Revisiting Prompt Engineering via Declarative Crowdsourcing Aditya G. Parameswaran Shreya Shankar Parth Asawa Naman Jain Yujie Wang 30 21 0 07 Aug 2023
WIKITIDE: A Wikipedia-Based Timestamped Definition Pairs Dataset Hsuvas Borkakoty Luis Espinosa-Anke 18 0 0 07 Aug 2023
Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text? Amrita Bhattacharjee Huang Liu DeLMO 17 56 0 02 Aug 2023
LP-MusicCaps: LLM-Based Pseudo Music Captioning Seungheon Doh Keunwoo Choi Jongpil Lee Juhan Nam 27 71 0 31 Jul 2023
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback Stephen Casper Xander Davies Claudia Shi T. Gilbert Jérémy Scheurer ... Erdem Biyik Anca Dragan David M. Krueger Dorsa Sadigh Dylan Hadfield-Menell ALM OffRL 42 470 0 27 Jul 2023
Leveraging Implicit Feedback from Deployment Data in Dialogue Richard Yuanzhe Pang Stephen Roller Kyunghyun Cho He He Jason Weston 46 7 0 26 Jul 2023
How to use LLMs for Text Analysis Petter Tornberg 19 10 0 24 Jul 2023
Wisdom of Instruction-Tuned Language Model Crowds. Exploring Model Label Variation Flor Miriam Plaza del Arco Debora Nozza Dirk Hovy ALM 40 4 0 24 Jul 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 88 10,977 0 18 Jul 2023
Towards a Neural Era in Dialogue Management for Collaboration: A Literature Survey Amogh Mannekote LLMAG 10 3 0 18 Jul 2023
Zero-shot Domain-sensitive Speech Recognition with Prompt-conditioning Fine-tuning F. Liao Yung-Chieh Chan Yi-Chang Chen Chan-Jan Hsu Da-shan Shiu 30 6 0 18 Jul 2023
AutoHint: Automatic Prompt Optimization with Hint Generation Hong Sun Xue Li Yi Xu Youkow Homma Qinhao Cao Min-man Wu Jian Jiao Denis Xavier Charles 29 23 0 13 Jul 2023
Can Large Language Models Aid in Annotating Speech Emotional Data? Uncovering New Frontiers S. Latif Muhammad Usama Mohammad Ibrahim Malik Björn W. Schuller 30 17 0 12 Jul 2023
UMASS_BioNLP at MEDIQA-Chat 2023: Can LLMs generate high-quality synthetic note-oriented doctor-patient conversations? Junda Wang Zonghai Yao Avijit Mitra Samuel Osebe Zhichao Yang Hongfeng Yu LM&MA MedIm 21 13 0 29 Jun 2023
Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Cost Parikshit Bansal Amit Sharma 36 40 0 27 Jun 2023
Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning Fuxiao Liu Kevin Qinghong Lin Linjie Li Jianfeng Wang Yaser Yacoob Lijuan Wang VLM MLLM 20 239 0 26 Jun 2023
LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding Robert F. Chew John Bollenbacher Michael Wenger Jessica Speer Annice Kim ELM 19 59 0 23 Jun 2023
System-Level Natural Language Feedback Weizhe Yuan Kyunghyun Cho Jason Weston 33 5 0 23 Jun 2023
Quilt-1M: One Million Image-Text Pairs for Histopathology Wisdom O. Ikezogwo M. S. Seyfioglu Fatemeh Ghezloo Dylan Stefan Chan Geva Fatwir Sheikh Mohammed Pavan Kumar Anand Ranjay Krishna Linda G. Shapiro CLIP VLM 133 112 0 20 Jun 2023
Leveraging ChatGPT As Text Annotation Tool For Sentiment Analysis Mohammad Belal James She Simon Wong AI4MH 17 30 0 18 Jun 2023
The Importance of Human-Labeled Data in the Era of LLMs Yang Liu ALM 15 8 0 18 Jun 2023
Explore, Establish, Exploit: Red Teaming Language Models from Scratch Stephen Casper Jason Lin Joe Kwon Gatlen Culp Dylan Hadfield-Menell AAML 8 83 0 15 Jun 2023
SIGHT: A Large Annotated Dataset on Student Insights Gathered from Higher Education Transcripts Rose E. Wang Pawan Wirawarn Noah D. Goodman Dorottya Demszky 30 11 0 15 Jun 2023
arXiVeri: Automatic table verification with GPT Gyungin Shin Weidi Xie Samuel Albanie LMTD 28 1 0 13 Jun 2023
Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks V. Veselovsky Manoel Horta Ribeiro Robert West 20 129 0 13 Jun 2023
Adding guardrails to advanced chatbots Yanchen Wang Lisa Singh AI4MH 15 7 0 13 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric P. Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 16 3,781 0 09 Jun 2023
Closing the Loop: Testing ChatGPT to Generate Model Explanations to Improve Human Labelling of Sponsored Content on Social Media Thales Bertaglia Stefan Huber Catalina Goanta Gerasimos Spanakis Adriana Iamnitchi 19 11 0 08 Jun 2023
K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization Cheng Deng Tianhang Zhang Zhongmou He Yi Xu Qiyuan Chen ... Weinan Zhang Xinbing Wang Cheng Zhou Zhouhan Lin Junxian He ALM 28 63 0 08 Jun 2023
Using Imperfect Surrogates for Downstream Inference: Design-based Supervised Learning for Social Science Applications of Large Language Models Naoki Egami Musashi Jacobs-Harukawa Brandon M Stewart Hanying Wei 22 14 0 07 Jun 2023
M $^3$ IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning Lei Li Yuwei Yin Shicheng Li Liang Chen Peiyi Wang ... Yazheng Yang Jingjing Xu Xu Sun Lingpeng Kong Qi Liu MLLM VLM 27 115 0 07 Jun 2023
GPT Self-Supervision for a Better Data Annotator Xiaohuan Pei Yanxi Li Chang Xu 30 7 0 07 Jun 2023
Is ChatGPT a Good Teacher Coach? Measuring Zero-Shot Performance For Scoring and Providing Actionable Insights on Classroom Instruction Rose E. Wang Dorottya Demszky 19 58 0 05 Jun 2023
Towards Coding Social Science Datasets with Language Models Anonymous Acl Taylor Sorensen Lisa P. Argyle Ethan C. Busby Nancy Fulda Joshua R Gubler David Wingate ALM SyDa 27 10 0 03 Jun 2023
Automated Annotation with Generative AI Requires Validation Nicholas Pangakis Samuel Wolken Neil Fasching SyDa 19 61 0 31 May 2023
GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction Rui Yang Lin Song Yanwei Li Sijie Zhao Yixiao Ge Xiu Li Ying Shan SyDa MLLM 26 208 0 30 May 2023
Training Socially Aligned Language Models on Simulated Social Interactions Ruibo Liu Ruixin Yang Chenyan Jia Ge Zhang Denny Zhou Andrew M. Dai Diyi Yang Soroush Vosoughi ALM 23 45 0 26 May 2023
Contrastive Learning of Sentence Embeddings from Scratch Junlei Zhang Zhenzhong Lan Junxian He SSL 20 21 0 24 May 2023
Annotation Imputation to Individualize Predictions: Initial Studies on Distribution Dynamics and Model Predictions London Lowmanstone Ruyuan Wan Risako Owan Jaehyung Kim Dongyeop Kang 17 1 0 24 May 2023
Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Science V. Veselovsky Manoel Horta Ribeiro Akhil Arora Martin Josifoski Ashton Anderson Robert West SyDa HILM 25 31 0 24 May 2023
GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP Md. Tawkat Islam Khondaker Abdul Waheed El Moatez Billah Nagoudi Muhammad Abdul-Mageed ELM LM&MA 24 62 0 24 May 2023
OverPrompt: Enhancing ChatGPT through Efficient In-Context Learning Jiazheng Li Runcong Zhao Yongxin Yang Yulan He Lin Gui 33 8 0 24 May 2023
ClusterLLM: Large Language Models as a Guide for Text Clustering Yuwei Zhang Zihan Wang Jingbo Shang 31 53 0 24 May 2023
Using Natural Language Explanations to Rescale Human Judgments Manya Wadhwa Jifan Chen Junyi Jessy Li Greg Durrett 33 8 0 24 May 2023
Modeling Empathic Similarity in Personal Narratives Jocelyn Shen Maarten Sap Pedro Colon-Hernandez Hae Won Park C. Breazeal 21 12 0 23 May 2023