Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary

1 October 2020

Papers citing "Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary"

27 / 27 papers shown

Title
Are LLM-generated plain language summaries truly understandable? A large-scale crowdsourced evaluation Yue Guo Jae Ho Sohn Gondy Leroy Trevor Cohen ELM 23 0 0 15 May 2025
RLPF: Reinforcement Learning from Prediction Feedback for User Summarization with LLMs Jiaxing Wu Lin Ning Luyang Liu Harrison Lee Neo Wu Chao Wang Sushant Prakash S. O’Banion Bradley Green Jun Xie 71 1 0 20 Jan 2025
Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs Jannik Kossen Jiatong Han Muhammed Razzak Lisa Schut Shreshth A. Malik Yarin Gal HILM 60 36 0 22 Jun 2024
Calibrating Likelihoods towards Consistency in Summarization Models Polina Zablotskaia Misha Khalman Rishabh Joshi Livio Baldini Soares Shoshana Jakobovits Joshua Maynez Shashi Narayan 31 3 0 12 Oct 2023
DecompEval: Evaluating Generated Texts as Unsupervised Decomposed Question Answering Pei Ke Fei Huang Fei Mi Yasheng Wang Qun Liu Xiaoyan Zhu Minlie Huang ReLM ELM 38 10 0 13 Jul 2023
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation Sewon Min Kalpesh Krishna Xinxi Lyu M. Lewis Wen-tau Yih Pang Wei Koh Mohit Iyyer Luke Zettlemoyer Hannaneh Hajishirzi HILM ALM 86 611 0 23 May 2023
APPLS: Evaluating Evaluation Metrics for Plain Language Summarization Yue Guo Tal August Gondy Leroy T. Cohen Lucy Lu Wang 57 9 0 23 May 2023
Zero-shot Faithful Factual Error Correction Kung-Hsiang Huang Hou Pong Chan Heng Ji KELM HILM 32 30 0 13 May 2023
Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation Yixin Liu Alexander R. Fabbri Yilun Zhao Pengfei Liu Chenyu You Chien-Sheng Wu Caiming Xiong Dragomir R. Radev 15 28 0 07 Mar 2023
On the State of German (Abstractive) Text Summarization Dennis Aumiller Jing Fan Michael Gertz 28 1 0 17 Jan 2023
Rethinking with Retrieval: Faithful Large Language Model Inference Hangfeng He Hongming Zhang Dan Roth KELM LRM 146 160 0 31 Dec 2022
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation Yixin Liu Alexander R. Fabbri Pengfei Liu Yilun Zhao Linyong Nan ... Simeng Han Chenyu You Chien-Sheng Wu Caiming Xiong Dragomir R. Radev ALM 31 133 0 15 Dec 2022
HaRiM $^+$ : Evaluating Summary Quality with Hallucination Risk Seonil Son Junsoo Park J. Hwang Junghwa Lee Hyungjong Noh Yeonsoo Lee HILM 19 8 0 22 Nov 2022
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization Ryo Kamoi Tanya Goyal Greg Durrett HILM 44 14 0 13 Oct 2022
Podcast Summary Assessment: A Resource for Evaluating Summary Assessment Methods Potsawee Manakul Mark Gales 15 5 0 28 Aug 2022
SMART: Sentences as Basic Units for Text Evaluation Reinald Kim Amplayo Peter J. Liu Yao-Min Zhao Shashi Narayan 38 21 0 01 Aug 2022
QASem Parsing: Text-to-text Modeling of QA-based Semantics Ayal Klein Eran Hirsch Ron Eliav Valentina Pyatkin Avi Caciularu Ido Dagan 49 12 0 23 May 2022
PREME: Preference-based Meeting Exploration through an Interactive Questionnaire Negar Arabzadeh Ali Ahmadvand Julia Kiseleva Yang Liu Ahmed Hassan Awadallah Ming Zhong Milad Shokouhi 28 4 0 05 May 2022
Repro: An Open-Source Library for Improving the Reproducibility and Usability of Publicly Available Research Code Daniel Deutsch Dan Roth AI4CE 45 2 0 29 Apr 2022
Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics Daniel Deutsch Rotem Dror Dan Roth 20 44 0 21 Apr 2022
Evaluation of Automatic Text Summarization using Synthetic Facts J. Ahn Foaad Khosmood HILM 18 0 0 11 Apr 2022
QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization Alexander R. Fabbri C. Wu Wenhao Liu Caiming Xiong HILM 22 208 0 16 Dec 2021
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection Sihao Chen Fan Zhang Kazoo Sone Dan Roth HILM 47 104 0 19 Apr 2021
What's in a Summary? Laying the Groundwork for Advances in Hospital-Course Summarization Griffin Adams Emily Alsentzer Mert Ketenci Jason Zucker Noémie Elhadad 54 47 0 12 Apr 2021
Understanding the Extent to which Summarization Evaluation Metrics Measure the Information Quality of Summaries Daniel Deutsch Dan Roth 61 7 0 23 Oct 2020
Summary-Oriented Question Generation for Informational Queries Xusen Yin Li Zhou Kevin Small Jonathan May 15 3 0 19 Oct 2020
SacreROUGE: An Open-Source Library for Using and Developing Summarization Evaluation Metrics Daniel Deutsch Dan Roth 22 26 0 10 Jul 2020