Asking and Answering Questions to Evaluate the Factual Consistency of Summaries

8 April 2020

Papers citing "Asking and Answering Questions to Evaluate the Factual Consistency of Summaries"

50 / 327 papers shown

Title
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling Vidhisha Balachandran Hannaneh Hajishirzi William W. Cohen Yulia Tsvetkov HILM KELM 92 46 0 22 Oct 2022
Analyzing and Evaluating Faithfulness in Dialogue Summarization Bin Wang Chen Zhang Yan Zhang Yiming Chen Haizhou Li HILM 41 15 0 21 Oct 2022
Taxonomy of Abstractive Dialogue Summarization: Scenarios, Approaches and Future Directions Qi Jia Yizhu Liu Siyu Ren Kenny Q. Zhu 29 6 0 18 Oct 2022
RARR: Researching and Revising What Language Models Say, Using Language Models Luyu Gao Zhuyun Dai Panupong Pasupat Anthony Chen Arun Tejasvi Chaganty ... Vincent Zhao Ni Lao Hongrae Lee Da-Cheng Juan Kelvin Guu HILM KELM 41 257 0 17 Oct 2022
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey Sachin Kumar Vidhisha Balachandran Lucille Njoo Antonios Anastasopoulos Yulia Tsvetkov ELM 77 85 0 14 Oct 2022
Towards a Unified Multi-Dimensional Evaluator for Text Generation Ming Zhong Yang Liu Da Yin Yuning Mao Yizhu Jiao Peng Liu Chenguang Zhu Heng Ji Jiawei Han ELM 45 255 0 13 Oct 2022
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization Ryo Kamoi Tanya Goyal Greg Durrett HILM 39 14 0 13 Oct 2022
Readability Controllable Biomedical Document Summarization Zheheng Luo Qianqian Xie Sophia Ananiadou 42 36 0 10 Oct 2022
Just ClozE! A Novel Framework for Evaluating the Factual Consistency Faster in Abstractive Summarization Yiyang Li Lei Li Marina Litvak N. Vanetik Dingxing Hu Yuze Li Yanquan Zhou HILM 40 0 0 06 Oct 2022
Probing of Quantitative Values in Abstractive Summarization Models Nathan M. White 15 0 0 03 Oct 2022
Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priors Vignav Ramesh Nathan Chi Pranav Rajpurkar MedIm 36 49 0 27 Sep 2022
MaXM: Towards Multilingual Visual Question Answering Soravit Changpinyo Linting Xue Michal Yarom Ashish V. Thapliyal Idan Szpektor J. Amelot Xi Chen Radu Soricut 33 8 0 12 Sep 2022
Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarization Shiyue Zhang David Wan Joey Tianyi Zhou HILM 52 27 0 08 Sep 2022
Podcast Summary Assessment: A Resource for Evaluating Summary Assessment Methods Potsawee Manakul Mark Gales 15 5 0 28 Aug 2022
Template-based Abstractive Microblog Opinion Summarisation I. Bilal Bo Wang Adam Tsakalidis Dong Nguyen Rob Procter M. Liakata 30 10 0 08 Aug 2022
SMART: Sentences as Basic Units for Text Evaluation Reinald Kim Amplayo Peter J. Liu Yao-Min Zhao Shashi Narayan 35 21 0 01 Aug 2022
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control Haopeng Zhang Semih Yavuz Wojciech Kry'sciñski Kazuma Hashimoto Yingbo Zhou HILM 38 34 0 05 Jul 2022
An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics Huan Yee Koh Jiaxin Ju Ming Liu Shirui Pan 81 122 0 03 Jul 2022
Conditional Generation with a Question-Answering Blueprint Shashi Narayan Joshua Maynez Reinald Kim Amplayo Kuzman Ganchev Annie Louis Fantine Huot Anders Sandholm Dipanjan Das Mirella Lapata 61 47 0 01 Jul 2022
MaskEval: Weighted MLM-Based Evaluation for Text Summarization and Simplification Yu Lu Liu Rachel Bawden Thomas Scaliom Benoît Sagot Jackie C.K. Cheung 35 4 0 24 May 2022
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization David Wan Joey Tianyi Zhou HILM 25 68 0 16 May 2022
Generating Literal and Implied Subquestions to Fact-check Complex Claims Jifan Chen Aniruddh Sriram Eunsol Choi Greg Durrett HILM 36 60 0 14 May 2022
Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarization Prasetya Ajie Utama Joshua Bambrick N. Moosavi Iryna Gurevych HILM 16 42 0 12 May 2022
ALIGNMEET: A Comprehensive Tool for Meeting Annotation, Alignment, and Evaluation Peter Polák Muskaan Singh A. Nedoluzhko Ondrej Bojar 23 9 0 11 May 2022
PREME: Preference-based Meeting Exploration through an Interactive Questionnaire Negar Arabzadeh Ali Ahmadvand Julia Kiseleva Yang Liu Ahmed Hassan Awadallah Ming Zhong Milad Shokouhi 25 4 0 05 May 2022
Masked Summarization to Generate Factually Inconsistent Summaries for Improved Factual Consistency Checking Hwanhee Lee Kang Min Yoo Joonsuk Park Hwaran Lee Kyomin Jung HILM 13 10 0 04 May 2022
All You May Need for VQA are Image Captions Soravit Changpinyo Doron Kukliansky Idan Szpektor Xi Chen Nan Ding Radu Soricut 32 70 0 04 May 2022
Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarization Yue Dong John Wieting Pat Verga HILM 24 24 0 28 Apr 2022
FaithDial: A Faithful Benchmark for Information-Seeking Dialogue Nouha Dziri Ehsan Kamalloo Sivan Milton Osmar Zaiane Mo Yu Edoardo Ponti Siva Reddy HILM 29 87 0 22 Apr 2022
Benchmarking Answer Verification Methods for Question Answering-Based Summarization Evaluation Metrics Daniel Deutsch Dan Roth 15 4 0 21 Apr 2022
Spurious Correlations in Reference-Free Evaluation of Text Generation Esin Durmus Faisal Ladhak Tatsunori Hashimoto 19 30 0 21 Apr 2022
A Survey on Neural Abstractive Summarization Methods and Factual Consistency of Summarization Meng Cao 10 6 0 20 Apr 2022
Evaluating Factuality in Text Simplification Ashwin Devaraj William Sheffield Byron C. Wallace Junyi Jessy Li HILM 27 41 0 15 Apr 2022
Summarization with Graphical Elements Maartje ter Hoeve Julia Kiseleva Maarten de Rijke 25 2 0 15 Apr 2022
Learning to Revise References for Faithful Summarization Griffin Adams Han-Chin Shing Q. Sun C. Winestock Kathleen McKeown Noémie Elhadad 19 32 0 13 Apr 2022
ASQA: Factoid Questions Meet Long-Form Answers Ivan Stelmakh Yi Luan Bhuwan Dhingra Ming-Wei Chang 32 160 0 12 Apr 2022
TRUE: Re-evaluating Factual Consistency Evaluation Or Honovich Roee Aharoni Jonathan Herzig Hagai Taitelbaum Doron Kukliansy Vered Cohen Thomas Scialom Idan Szpektor Avinatan Hassidim Yossi Matias HILM 35 3 0 11 Apr 2022
Evaluation of Automatic Text Summarization using Synthetic Facts J. Ahn Foaad Khosmood HILM 18 0 0 11 Apr 2022
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering Wenhu Chen Pat Verga Michiel de Jong John Wieting William W. Cohen RALM KELM 32 26 0 10 Apr 2022
Quality Assurance of Generative Dialog Models in an Evolving Conversational Agent Used for Swedish Language Practice Markus Borg J. Bengtsson Harald Österling Alexander Hagelborn Isabella Gagner Piotr Tomaszewski 14 1 0 29 Mar 2022
Probing Factually Grounded Content Transfer with Factual Ablation Peter West Chris Quirk Michel Galley Yejin Choi HILM 30 9 0 18 Mar 2022
Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search Daniel King Zejiang Shen Nishant Subramani Daniel S. Weld Iz Beltagy Doug Downey HILM 28 31 0 16 Mar 2022
Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods Wei Li Wenhao Wu Moye Chen Jiachen Liu Xinyan Xiao Hua Wu HILM 23 27 0 10 Mar 2022
On the Evaluation of Answer-Agnostic Paragraph-level Multi-Question Generation Jishnu Ray Chowdhury Debanjan Mahata Cornelia Caragea 30 2 0 09 Mar 2022
Read before Generate! Faithful Long Form Question Answering with Machine Reading Dan Su Xiaoguang Li Jindi Zhang Lifeng Shang Xin Jiang Qun Liu Pascale Fung HILM 19 59 0 01 Mar 2022
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text Sebastian Gehrmann Elizabeth Clark Thibault Sellam ELM AI4CE 69 184 0 14 Feb 2022
Survey of Hallucination in Natural Language Generation Ziwei Ji Nayeon Lee Rita Frieske Tiezheng Yu D. Su ... Delong Chen Wenliang Dai Ho Shu Chan Andrea Madotto Pascale Fung HILM LRM 73 2,248 0 08 Feb 2022
New Methods & Metrics for LFQA tasks Suchismit Mahapatra V. Blagojević P. Bertorello Prasanna Kumar 25 2 0 26 Dec 2021
Measuring Attribution in Natural Language Generation Models Hannah Rashkin Vitaly Nikolaev Matthew Lamm Lora Aroyo Michael Collins Dipanjan Das Slav Petrov Gaurav Singh Tomar Iulia Turc David Reitter 39 173 0 23 Dec 2021
Consistency and Coherence from Points of Contextual Similarity Oleg V. Vasilyev John Bohannon HILM 33 1 0 22 Dec 2021