Asking and Answering Questions to Evaluate the Factual Consistency of Summaries

8 April 2020

Papers citing "Asking and Answering Questions to Evaluate the Factual Consistency of Summaries"

50 / 327 papers shown

Title
CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuning Xiangru Tang Arjun Nair Borui Wang Bingyao Wang Jai Desai Aaron Wade Haoran Li Asli Celikyilmaz Yashar Mehdad Dragomir R. Radev HILM 25 62 0 16 Dec 2021
QuALITY: Question Answering with Long Input Texts, Yes! Richard Yuanzhe Pang Alicia Parrish Nitish Joshi Nikita Nangia Jason Phang ... Vishakh Padmakumar Johnny Ma Jana Thompson He He Sam Bowman RALM 30 141 0 16 Dec 2021
QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization Alexander R. Fabbri C. Wu Wenhao Liu Caiming Xiong HILM 17 207 0 16 Dec 2021
CO2Sum:Contrastive Learning for Factual-Consistent Abstractive Summarization Wei Liu Huanqin Wu Wenjing Mu Zhen Li Tao Chen Dan Nie HILM 14 17 0 02 Dec 2021
Incorporating Question Answering-Based Signals into Abstractive Summarization via Salient Span Selection Daniel Deutsch Dan Roth 10 6 0 15 Nov 2021
TODSum: Task-Oriented Dialogue Summarization with State Tracking Lulu Zhao Fujia Zheng Keqing He Weihao Zeng Yuejie Lei Huixing Jiang Wei Wu Weiran Xu Jun Guo Fanyu Meng 42 23 0 25 Oct 2021
CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization Prafulla Kumar Choubey Alexander R. Fabbri Jesse Vig Chien-Sheng Wu Wenhao Liu Nazneen Rajani HILM 24 16 0 14 Oct 2021
Explainable Fact-checking through Question Answering Jing Yang D. Vega-Oliveros Taís Seibt Anderson de Rezende Rocha HILM 27 14 0 11 Oct 2021
Finding a Balanced Degree of Automation for Summary Evaluation Shiyue Zhang Joey Tianyi Zhou 55 43 0 23 Sep 2021
Recursively Summarizing Books with Human Feedback Jeff Wu Long Ouyang Daniel M. Ziegler Nissan Stiennon Ryan J. Lowe Jan Leike Paul Christiano ALM 35 295 0 22 Sep 2021
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization Shuyang Cao Lu Wang HILM 31 175 0 19 Sep 2021
Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries Xiangru Tang Alexander R. Fabbri Haoran Li Ziming Mao Griffin Adams Borui Wang Asli Celikyilmaz Yashar Mehdad Dragomir R. Radev HILM 13 19 0 19 Sep 2021
Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generation Mingkai Deng Bowen Tan Zhengzhong Liu Eric Xing Zhiting Hu 16 72 0 14 Sep 2021
Learning Opinion Summarizers by Selecting Informative Reviews Arthur Brazinskas Mirella Lapata Ivan Titov 53 29 0 09 Sep 2021
TruthfulQA: Measuring How Models Mimic Human Falsehoods Stephanie C. Lin Jacob Hilton Owain Evans HILM 57 1,742 0 08 Sep 2021
Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization Faisal Ladhak Esin Durmus He He Claire Cardie Kathleen McKeown 14 64 0 31 Aug 2021
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization Mengyao Cao Yue Dong Jackie C.K. Cheung HILM 178 146 0 30 Aug 2021
Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation Yuexiang Xie Fei Sun Yang Deng Yaliang Li Bolin Ding HILM 26 53 0 30 Aug 2021
QACE: Asking Questions to Evaluate an Image Caption Hwanhee Lee Thomas Scialom Seunghyun Yoon Franck Dernoncourt Kyomin Jung CoGe 25 18 0 28 Aug 2021
Evaluating the Tradeoff Between Abstractiveness and Factuality in Abstractive Summarization Markus Dreyer Mengwen Liu Feng Nan Sandeep Atluri Sujith Ravi HILM 51 16 0 05 Aug 2021
EmailSum: Abstractive Email Thread Summarization Shiyue Zhang Asli Celikyilmaz Jianfeng Gao Joey Tianyi Zhou 30 38 0 30 Jul 2021
Keep it Simple: Unsupervised Simplification of Multi-Paragraph Text Philippe Laban Tobias Schnabel Paul N. Bennett Marti A. Hearst 21 60 0 07 Jul 2021
Improving Factual Consistency of Abstractive Summarization on Customer Feedback Yang Liu Yifei Sun Vincent Gao HILM 18 6 0 30 Jun 2021
XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages Tahmid Hasan Abhik Bhattacharjee Md. Saiful Islam Kazi Samin Mubasshir Yuan-Fang Li Yong-Bin Kang M. Rahman Rifat Shahriyar 37 344 0 25 Jun 2021
BARTScore: Evaluating Generated Text as Text Generation Weizhe Yuan Graham Neubig Pengfei Liu 47 808 0 22 Jun 2021
Automatic Document Sketching: Generating Drafts from Analogous Texts Zeqiu Wu Michel Galley Chris Brockett Yizhe Zhang Bill Dolan 52 5 0 14 Jun 2021
AgreeSum: Agreement-Oriented Multi-Document Summarization Richard Yuanzhe Pang Á. Lelkes Vinh Q. Tran Cong Yu 13 16 0 04 Jun 2021
Zero-shot Fact Verification by Claim Generation Liangming Pan Wenhu Chen Wenhan Xiong Min-Yen Kan Wei Wang 34 56 0 31 May 2021
Focus Attention: Promoting Faithfulness and Diversity in Summarization Rahul Aralikatte Shashi Narayan Joshua Maynez S. Rothe Ryan T. McDonald 35 45 0 25 May 2021
Improving Factual Consistency of Abstractive Summarization via Question Answering Feng Nan Cicero Nogueira dos Santos Henghui Zhu Patrick Ng Kathleen McKeown Ramesh Nallapati Dejiao Zhang Zhiguo Wang Andrew O. Arnold Bing Xiang HILM 14 82 0 10 May 2021
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark Nouha Dziri Hannah Rashkin Tal Linzen David Reitter ALM 195 79 0 30 Apr 2021
The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey Yi-Chong Huang Xiachong Feng Xiaocheng Feng Bing Qin HILM 136 105 0 30 Apr 2021
Towards Clinical Encounter Summarization: Learning to Compose Discharge Summaries from Prior Notes Han-Chin Shing Chaitanya P. Shivade Nima Pourdamghani Feng Nan Philip Resnik Douglas W. Oard Parminder Bhatia 60 24 0 27 Apr 2021
Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics Artidoro Pagnoni Vidhisha Balachandran Yulia Tsvetkov HILM 231 306 0 27 Apr 2021
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation Tianyu Liu Yizhe Zhang Chris Brockett Yi Mao Zhifang Sui Weizhu Chen W. Dolan HILM 228 144 0 18 Apr 2021
Multi-Perspective Abstractive Answer Summarization Alexander R. Fabbri Xiaojian Wu Srini Iyer Mona T. Diab 19 6 0 17 Apr 2021
Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding Nouha Dziri Andrea Madotto Osmar Zaiane A. Bose HILM 28 132 0 17 Apr 2021
$$Q^{2}$: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering$ $Q^{2}$ : Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering Or Honovich Leshem Choshen Roee Aharoni Ella Neeman Idan Szpektor Omri Abend HILM 36 138 0 16 Apr 2021
MS2: Multi-Document Summarization of Medical Studies Jay DeYoung Iz Beltagy Madeleine van Zuylen Bailey Kuehl Lucy Lu Wang 15 107 0 13 Apr 2021
Estimation of Summary-to-Text Inconsistency by Mismatched Embeddings Oleg V. Vasilyev John Bohannon HILM 28 7 0 12 Apr 2021
Annotating and Modeling Fine-grained Factuality in Summarization Tanya Goyal Greg Durrett HILM 21 153 0 09 Apr 2021
Efficient Attentions for Long Document Summarization L. Huang Shuyang Cao Nikolaus Nova Parulian Heng Ji Lu Wang 67 273 0 05 Apr 2021
Paired Examples as Indirect Supervision in Latent Decision Models Nitish Gupta Sameer Singh Matt Gardner Dan Roth 23 7 0 05 Apr 2021
A New Approach to Overgenerating and Scoring Abstractive Summaries Kaiqiang Song Bingqing Wang Z. Feng Fei Liu 22 17 0 05 Apr 2021
QuestEval: Summarization Asks for Fact-based Evaluation Thomas Scialom Paul-Alexis Dray Patrick Gallinari Sylvain Lamprier Benjamin Piwowarski Jacopo Staiano Alex Jinpeng Wang HILM 16 267 0 23 Mar 2021
Hurdles to Progress in Long-form Question Answering Kalpesh Krishna Aurko Roy Mohit Iyyer 25 192 0 10 Mar 2021
Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View Tianyu Liu Xin Zheng Baobao Chang Zhifang Sui 127 35 0 17 Feb 2021
Toward Improving Coherence and Diversity of Slogan Generation Yiping Jin Akshay Bhatia Dittaya Wanvarie Phu T. V. Le 19 5 0 11 Feb 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics Sebastian Gehrmann Tosin P. Adewumi Karmanya Aggarwal Pawan Sasanka Ammanamanchi Aremu Anuoluwapo ... Nishant Subramani Wei-ping Xu Diyi Yang Akhila Yerukola Jiawei Zhou VLM 260 285 0 02 Feb 2021
What Makes a Good and Useful Summary? Incorporating Users in Automatic Summarization Research Maartje ter Hoeve Julia Kiseleva Maarten de Rijke 33 7 0 14 Dec 2020