Human-like Summarization Evaluation with ChatGPT

5 April 2023

Xiaojun Wan

Papers citing "Human-like Summarization Evaluation with ChatGPT"

29 / 29 papers shown

Title
WaLLM -- Insights from an LLM-Powered Chatbot deployment via WhatsApp Hiba Eltigani Rukhshan Haroon Asli Kocak Abdullah Bin Faisal Noah Martin Fahad Dogar 19 0 0 13 May 2025
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong Xuzhao Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Tianwei Zhang ALM ELM 86 2 0 26 Apr 2025
Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts Yifei Yu Qian Zhang Lingfeng Qiao Di Yin Fang Li Jie Wang Z. Chen Suncong Zheng Xiaolong Liang Xingchen Sun 44 0 0 07 Apr 2025
Unequal Opportunities: Examining the Bias in Geographical Recommendations by Large Language Models Shiran Dudy Thulasi Tholeti R. Ramachandranpillai Muhammad Ali Toby Jia-Jun Li Ricardo Baeza-Yates 29 0 0 16 Mar 2025
BPO: Towards Balanced Preference Optimization between Knowledge Breadth and Depth in Alignment Sizhe Wang Yongqi Tong Hengyuan Zhang Dawei Li Xin Zhang Tianlong Chen 85 5 0 21 Feb 2025
LAMD: Context-driven Android Malware Detection and Classification with LLMs Xingzhi Qian Xinran Zheng Yiling He Shuo Yang Lorenzo Cavallaro 83 2 0 18 Feb 2025
Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performance Borui Xu Yao Chen Zeyi Wen Weiguo Liu Bingsheng He 79 1 0 02 Feb 2025
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge Dawei Li Bohan Jiang Liangjie Huang Alimohammad Beigi Chengshuai Zhao ... Canyu Chen Tianhao Wu Kai Shu Lu Cheng Huan Liu ELM AILaw 123 70 0 25 Nov 2024
Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Jing Jiang Min-Bin Lin 47 8 0 09 Oct 2024
Hard Prompts Made Interpretable: Sparse Entropy Regularization for Prompt Tuning with RL Yunseon Choi Sangmin Bae Seonghyun Ban Minchan Jeong Chuheng Zhang Lei Song Li Zhao Jiang Bian Kee-Eung Kim VLM AAML 36 3 0 20 Jul 2024
Cohesive Conversations: Enhancing Authenticity in Multi-Agent Simulated Dialogues Kuanchao Chu Yi-Pei Chen Hideki Nakayama LLMAG 42 2 0 13 Jul 2024
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations Md Tahmid Rahman Laskar Sawsan Alqahtani M Saiful Bari Mizanur Rahman Mohammad Abdullah Matin Khan ... Chee Wei Tan Md. Rizwan Parvez Enamul Hoque Chenyu You Jimmy Huang ELM ALM 31 28 0 04 Jul 2024
FineSurE: Fine-grained Summarization Evaluation using LLMs Hwanjun Song Hang Su Igor Shalyminov Jason (Jinglun) Cai Saab Mansour HILM 41 31 0 01 Jul 2024
Unlearning Climate Misinformation in Large Language Models Michael Fore Simranjit Singh Chaehong Lee Amritanshu Pandey Antonios Anastasopoulos Dimitrios Stamoulis MU 62 1 0 29 May 2024
Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models Zhangyue Yin Qiushi Sun Qipeng Guo Zhiyuan Zeng Xiaonan Li ... Qinyuan Cheng Ding Wang Xiaofeng Mou Xipeng Qiu XuanJing Huang LRM 46 4 0 21 May 2024
RepEval: Effective Text Evaluation with LLM Representation Shuqian Sheng Yi Xu Tianhang Zhang Zanwei Shen Luoyi Fu Jiaxin Ding Lei Zhou Xinbing Wang Cheng Zhou 27 1 0 30 Apr 2024
A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods Hanlei Jin Yang Zhang Dan Meng Jun Wang Jinghua Tan 68 80 0 05 Mar 2024
Prediction-Powered Ranking of Large Language Models Ivi Chatzi Eleni Straitouri Suhas Thejaswi Manuel Gomez Rodriguez ALM 29 5 0 27 Feb 2024
Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs Simone Balloccu Patrícia Schmidtová Mateusz Lango Ondrej Dusek SILM ELM PILM 26 158 0 06 Feb 2024
LLM-based NLG Evaluation: Current Status and Challenges Mingqi Gao Xinyu Hu Jie Ruan Xiao Pu Xiaojun Wan ELM LM&MA 65 29 0 02 Feb 2024
OpinSummEval: Revisiting Automated Evaluation for Opinion Summarization Yuchen Shen Xiaojun Wan 38 9 0 27 Oct 2023
Language Models Hallucinate, but May Excel at Fact Verification Jian Guan Jesse Dodge David Wadden Minlie Huang Hao Peng LRM HILM 34 28 0 23 Oct 2023
Tuna: Instruction Tuning using Feedback from Large Language Models Haoran Li Yiran Liu Xingxing Zhang Wei Lu Furu Wei ALM 38 3 0 20 Oct 2023
Improving Summarization with Human Edits Zonghai Yao Benjamin J Schloss Sai P. Selvaraj 29 3 0 09 Oct 2023
AlignScore: Evaluating Factual Consistency with a Unified Alignment Function Yuheng Zha Yichi Yang Ruichen Li Zhiting Hu HILM 21 180 0 26 May 2023
APPLS: Evaluating Evaluation Metrics for Plain Language Summarization Yue Guo Tal August Gondy Leroy T. Cohen Lucy Lu Wang 57 9 0 23 May 2023
Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization Chenhui Shen Liying Cheng Xuan-Phi Nguyen Yang You Lidong Bing ELM ALM 47 64 0 22 May 2023
DocAsRef: An Empirical Study on Repurposing Reference-Based Summary Quality Metrics Reference-Freely F. S. Bao Ruixuan Tu Ge Luo Yinfei Yang Hebi Li Minghui Qiu Youbiao He Cen Chen 21 2 0 20 Dec 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 339 12,003 0 04 Mar 2022