Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation

30 August 2021

Papers citing "Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation"

36 / 36 papers shown

Title
From Speech to Summary: A Comprehensive Survey of Speech Summarization Fabian Retkowski Maike Züfle Andreas Sudmann Dinah Pfau Jan Niehues Alexander Waibel 44 0 0 10 Apr 2025
An Empirical Comparison of Text Summarization: A Multi-Dimensional Evaluation of Large Language Models Anantharaman Janakiraman Behnaz Ghoraani ELM HILM 42 0 0 06 Apr 2025
Fine-grained and Explainable Factuality Evaluation for Multimodal Summarization Liqiang Jing Jingxuan Zuo Yue Zhang 47 7 0 31 Dec 2024
Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance Omer Nahum Nitay Calderon Orgad Keller Idan Szpektor Roi Reichart 25 2 0 24 Oct 2024
Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework Lu Chen Ruqing Zhang Jiafeng Guo Yixing Fan Xueqi Cheng 29 3 0 24 Sep 2024
Temporally Consistent Factuality Probing for Large Language Models Ashutosh Bajpai Aaryan Goyal Atif Anwer Tanmoy Chakraborty HILM 15 0 0 21 Sep 2024
Natural Language Processing RELIES on Linguistics Juri Opitz Shira Wein Nathan Schneider AI4CE 55 7 0 09 May 2024
Can We Catch the Elephant? A Survey of the Evolvement of Hallucination Evaluation on Natural Language Generation Siya Qi Yulan He Zheng Yuan LRM HILM 43 1 0 18 Apr 2024
A Unified Causal View of Instruction Tuning Luyao Chen Wei Huang Ruqing Zhang Wei Chen J. Guo Xueqi Cheng 28 1 0 09 Feb 2024
Zero-shot Faithfulness Evaluation for Text Summarization with Foundation Language Model Qi Jia Siyu Ren Yizhu Liu Kenny Q. Zhu ALM HILM 33 16 0 18 Oct 2023
Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignment Boyang Xue Weichao Wang Hongru Wang Fei Mi Rui Wang Yasheng Wang Lifeng Shang Xin Jiang Qun Liu Kam-Fai Wong KELM HILM 219 15 0 12 Oct 2023
Inducing Causal Structure for Abstractive Text Summarization Luyao Chen Ruqing Zhang Wei Huang Wei Chen J. Guo Xueqi Cheng CML 21 1 0 24 Aug 2023
Evaluating Factual Consistency of Texts with Semantic Role Labeling Jing Fan Dennis Aumiller Michael Gertz HILM 34 4 0 22 May 2023
Counterfactual Debiasing for Generating Factually Consistent Text Summaries Chenhe Dong Yuexiang Xie Yaliang Li Ying Shen CML HILM 29 0 0 18 May 2023
FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge Shangbin Feng Vidhisha Balachandran Yuyang Bai Yulia Tsvetkov KELM HILM 23 52 0 14 May 2023
Automatically Summarizing Evidence from Clinical Trials: A Prototype Highlighting Current Challenges S. Ramprasad Denis Jered McInerney Iain J. Marshal Byron C. Wallace 29 9 0 07 Mar 2023
The Exploration of Knowledge-Preserving Prompts for Document Summarisation Cheng Chen Wei Emma Zhang A. Shakeri Makhmoor Fiza KELM 27 0 0 27 Jan 2023
BUMP: A Benchmark of Unfaithful Minimal Pairs for Meta-Evaluation of Faithfulness Metrics Liang Ma Shuyang Cao IV RobertL.Logan Di Lu Shihao Ran Kecheng Zhang Joel R. Tetreault A. Jaimes 17 6 0 20 Dec 2022
Improving Faithfulness of Abstractive Summarization by Controlling Confounding Effect of Irrelevant Sentences Asish Ghoshal Arash Einolghozati A. Arun Haoran Li L. Yu Vera Gor Yashar Mehdad Scott Yih Asli Celikyilmaz HILM 29 1 0 19 Dec 2022
CoP: Factual Inconsistency Detection by Controlling the Preference Shuaijie She Xiang Geng Shujian Huang Jiajun Chen 25 4 0 03 Dec 2022
HaRiM $^+$ : Evaluating Summary Quality with Hallucination Risk Seonil Son Junsoo Park J. Hwang Junghwa Lee Hyungjong Noh Yeonsoo Lee HILM 16 8 0 22 Nov 2022
Abstractive Summarization Guided by Latent Hierarchical Document Structure Yifu Qiu Shay B. Cohen 31 12 0 17 Nov 2022
Evaluating and Improving Factuality in Multimodal Abstractive Summarization David Wan Joey Tianyi Zhou 18 10 0 04 Nov 2022
How Far are We from Robust Long Abstractive Summarization? Huan Yee Koh Jiaxin Ju He Zhang Ming Liu Shirui Pan HILM 25 39 0 30 Oct 2022
Precisely the Point: Adversarial Augmentations for Faithful and Informative Text Generation Wenhao Wu Wei Li Jiachen Liu Xinyan Xiao Sujian Li Yajuan Lyu 39 3 0 22 Oct 2022
Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarization Shiyue Zhang David Wan Joey Tianyi Zhou HILM 52 27 0 08 Sep 2022
Masked Summarization to Generate Factually Inconsistent Summaries for Improved Factual Consistency Checking Hwanhee Lee Kang Min Yoo Joonsuk Park Hwaran Lee Kyomin Jung HILM 13 10 0 04 May 2022
Knowledge-aware Document Summarization: A Survey of Knowledge, Embedding Methods and Architectures Yutong Qu Wei Emma Zhang Jian Yang Lingfei Wu Jia Wu AI4TS 17 7 0 24 Apr 2022
Learning to Revise References for Faithful Summarization Griffin Adams Han-Chin Shing Q. Sun C. Winestock Kathleen McKeown Noémie Elhadad 19 32 0 13 Apr 2022
TRUE: Re-evaluating Factual Consistency Evaluation Or Honovich Roee Aharoni Jonathan Herzig Hagai Taitelbaum Doron Kukliansy Vered Cohen Thomas Scialom Idan Szpektor Avinatan Hassidim Yossi Matias HILM 35 3 0 11 Apr 2022
Probing Factually Grounded Content Transfer with Factual Ablation Peter West Chris Quirk Michel Galley Yejin Choi HILM 30 9 0 18 Mar 2022
Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods Wei Li Wenhao Wu Moye Chen Jiachen Liu Xinyan Xiao Hua-Hong Wu HILM 23 27 0 10 Mar 2022
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text Sebastian Gehrmann Elizabeth Clark Thibault Sellam ELM AI4CE 69 183 0 14 Feb 2022
New Methods & Metrics for LFQA tasks Suchismit Mahapatra V. Blagojević P. Bertorello Prasanna Kumar 19 2 0 26 Dec 2021
What Makes a Good and Useful Summary? Incorporating Users in Automatic Summarization Research Maartje ter Hoeve Julia Kiseleva Maarten de Rijke 33 7 0 14 Dec 2020
Text Summarization with Pretrained Encoders Yang Liu Mirella Lapata MILM 258 1,432 0 22 Aug 2019