Is GPT-4 a reliable rater? Evaluating Consistency in GPT-4 Text Ratings

Is GPT-4 a reliable rater? Evaluating Consistency in GPT-4 Text Ratings

3 August 2023

Alexandra Elena Müller

Michael Granitzer

Maximilian Sailer

Papers citing "Is GPT-4 a reliable rater? Evaluating Consistency in GPT-4 Text Ratings"

10 / 10 papers shown

Title
Evaluation Under Imperfect Benchmarks and Ratings: A Case Study in Text Simplification Joseph Liu Yoonsoo Nam Xinyue Cui Swabha Swayamdipta 56 0 0 13 Apr 2025
A Scalable Framework for Evaluating Health Language Models Neil Mallinar A. Heydari Xin Liu Anthony Z. Faranesh Brent Winslow ... Mark Malhotra Shwetak N. Patel Javier L. Prieto Daniel J. McDuff Ahmed A. Metwally LM&MA 64 2 0 30 Mar 2025
Reward-RAG: Enhancing RAG with Reward Driven Supervision Thang Nguyen Peter Chin Yu-Wing Tai RALM 45 4 0 03 Oct 2024
CHARP: Conversation History AwaReness Probing for Knowledge-grounded Dialogue Systems Abbas Ghaddar David Alfonso-Hermelo Philippe Langlais Mehdi Rezagholizadeh Boxing Chen Prasanna Parthasarathi 39 0 0 24 May 2024
Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models Zhangyue Yin Qiushi Sun Qipeng Guo Zhiyuan Zeng Xiaonan Li ... Qinyuan Cheng Ding Wang Xiaofeng Mou Xipeng Qiu XuanJing Huang LRM 46 4 0 21 May 2024
Automating Customer Needs Analysis: A Comparative Study of Large Language Models in the Travel Industry Simone Barandoni F. Chiarello Lorenzo Cascone Emiliano Marrale Salvatore Puccio 51 5 0 27 Apr 2024
LLM Evaluators Recognize and Favor Their Own Generations Arjun Panickssery Samuel R. Bowman Shi Feng 52 159 0 15 Apr 2024
StruQ: Defending Against Prompt Injection with Structured Queries Sizhe Chen Julien Piet Chawin Sitawarin David Wagner SILM AAML 35 68 0 09 Feb 2024
Learning to Perform Complex Tasks through Compositional Fine-Tuning of Language Models Victor S. Bursztyn David Demeter Doug Downey Larry Birnbaum ReLM LRM 48 8 0 23 Oct 2022
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 272 347 0 01 Feb 2021