Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics

27 April 2021

Artidoro Pagnoni

Vidhisha Balachandran

Yulia Tsvetkov

HILM

ArXiv PDF HTML

Papers citing "Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics"

25 / 75 papers shown

Title
Analyzing and Evaluating Faithfulness in Dialogue Summarization Bin Wang Chen Zhang Yan Zhang Yiming Chen Haizhou Li HILM 41 14 0 21 Oct 2022
Model Criticism for Long-Form Text Generation Yuntian Deng Volodymyr Kuleshov Alexander M. Rush 38 19 0 16 Oct 2022
Just ClozE! A Novel Framework for Evaluating the Factual Consistency Faster in Abstractive Summarization Yiyang Li Lei Li Marina Litvak N. Vanetik Dingxing Hu Yuze Li Yanquan Zhou HILM 40 0 0 06 Oct 2022
An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics Huan Yee Koh Jiaxin Ju Ming Liu Shirui Pan 75 122 0 03 Jul 2022
Gradient-Based Constrained Sampling from Language Models Sachin Kumar Biswajit Paria Yulia Tsvetkov BDL 30 53 0 25 May 2022
SQuALITY: Building a Long-Document Summarization Dataset the Hard Way Alex Jinpeng Wang Richard Yuanzhe Pang Angelica Chen Jason Phang Samuel R. Bowman 74 44 0 23 May 2022
Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets Philippe Laban Chien-Sheng Wu Wenhao Liu Caiming Xiong 38 5 0 13 May 2022
Masked Summarization to Generate Factually Inconsistent Summaries for Improved Factual Consistency Checking Hwanhee Lee Kang Min Yoo Joonsuk Park Hwaran Lee Kyomin Jung HILM 13 10 0 04 May 2022
QRelScore: Better Evaluating Generated Questions with Deeper Understanding of Context-aware Relevance Xiaoqiang Wang Bang Liu Siliang Tang Lingfei Wu 30 9 0 29 Apr 2022
Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarization Yue Dong John Wieting Pat Verga HILM 24 24 0 28 Apr 2022
Evaluating Factuality in Text Simplification Ashwin Devaraj William Sheffield Byron C. Wallace Junyi Jessy Li HILM 21 41 0 15 Apr 2022
Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters Tal Schuster Sihao Chen S. Buthpitiya Alex Fabrikant Donald Metzler 20 41 0 15 Apr 2022
NeuS: Neutral Multi-News Summarization for Mitigating Framing Bias Nayeon Lee Yejin Bang Tiezheng Yu Andrea Madotto Pascale Fung 17 24 0 11 Apr 2022
Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search Daniel King Zejiang Shen Nishant Subramani Daniel S. Weld Iz Beltagy Doug Downey HILM 20 31 0 16 Mar 2022
Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processing Tal August Lucy Lu Wang Jonathan Bragg Marti A. Hearst Andrew Head Kyle Lo 17 40 0 28 Feb 2022
Survey of Hallucination in Natural Language Generation Ziwei Ji Nayeon Lee Rita Frieske Tiezheng Yu D. Su ... Delong Chen Wenliang Dai Ho Shu Chan Andrea Madotto Pascale Fung HILM LRM 49 2,234 0 08 Feb 2022
CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuning Xiangru Tang Arjun Nair Borui Wang Bingyao Wang Jai Desai Aaron Wade Haoran Li Asli Celikyilmaz Yashar Mehdad Dragomir R. Radev HILM 19 62 0 16 Dec 2021
Explainable Fact-checking through Question Answering Jing Yang D. Vega-Oliveros Taís Seibt Anderson de Rezende Rocha HILM 19 14 0 11 Oct 2021
Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries Xiangru Tang Alexander R. Fabbri Haoran Li Ziming Mao Griffin Adams Borui Wang Asli Celikyilmaz Yashar Mehdad Dragomir R. Radev HILM 13 19 0 19 Sep 2021
ComSum: Commit Messages Summarization and Meaning Preservation Leshem Choshen Idan Amit 17 4 0 23 Aug 2021
Controlled Text Generation as Continuous Optimization with Multiple Constraints Sachin Kumar Eric Malmi Aliaksei Severyn Yulia Tsvetkov BDL AI4CE 43 76 0 04 Aug 2021
Underreporting of errors in NLG output, and what to do about it Emiel van Miltenburg Miruna Clinciu Ondrej Dusek Dimitra Gkatzia Stephanie Inglis ... Saad Mahamood Emma Manning S. Schoch Craig Thomson Luou Wen 27 38 0 02 Aug 2021
Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation Prakhar Gupta Yulia Tsvetkov Jeffrey P. Bigham 34 22 0 10 Jun 2021
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark Nouha Dziri Hannah Rashkin Tal Linzen David Reitter ALM 192 79 0 30 Apr 2021
Text Summarization with Pretrained Encoders Yang Liu Mirella Lapata MILM 258 1,432 0 22 Aug 2019