GO FIGURE: A Meta Evaluation of Factuality in Summarization

24 October 2020

Yejin Choi

Papers citing "GO FIGURE: A Meta Evaluation of Factuality in Summarization"

33 / 33 papers shown

Title
SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation Song Duong Florian Le Bronnec Alexandre Allauzen Vincent Guigue Alberto Lumbreras Laure Soulier Patrick Gallinari HILM 50 0 0 20 Feb 2025
DHP Benchmark: Are LLMs Good NLG Evaluators? Yicheng Wang Jiayi Yuan Yu-Neng Chuang Zhuoer Wang Yingchi Liu Mark Cusick Param Kulkarni Zhengping Ji Yasser Ibrahim Xia Hu LM&MA ELM 47 3 0 25 Aug 2024
STORYSUMM: Evaluating Faithfulness in Story Summarization Melanie Subbiah Faisal Ladhak Akankshya Mishra Griffin Adams Lydia B. Chilton Kathleen McKeown 48 4 0 09 Jul 2024
Factual Dialogue Summarization via Learning from Large Language Models Rongxin Zhu Jey Han Lau Jianzhong Qi HILM 52 1 0 20 Jun 2024
A Modular Approach for Multimodal Summarization of TV Shows Louis Mahon Mirella Lapata 26 9 0 06 Mar 2024
Rethinking Scientific Summarization Evaluation: Grounding Explainable Metrics on Facet-aware Benchmark Xiuying Chen Tairan Wang Qingqing Zhu Taicheng Guo Shen Gao Zhiyong Lu Xin Gao Xiangliang Zhang 75 2 0 22 Feb 2024
Named Entity Inclusion in Abstractive Text Summarization S. Berezin Tatiana Batura 36 7 0 05 Jul 2023
MuLER: Detailed and Scalable Reference-based Evaluation Taelin Karidi Leshem Choshen Gal Patel Omri Abend 30 0 0 24 May 2023
APPLS: Evaluating Evaluation Metrics for Plain Language Summarization Yue Guo Tal August Gondy Leroy T. Cohen Lucy Lu Wang 57 9 0 23 May 2023
Text-Blueprint: An Interactive Platform for Plan-based Conditional Generation Fantine Huot Joshua Maynez Shashi Narayan Reinald Kim Amplayo Kuzman Ganchev Annie Louis Anders Sandholm Dipanjan Das Mirella Lapata 24 7 0 28 Apr 2023
Lay Text Summarisation Using Natural Language Processing: A Narrative Literature Review Oliver Vinzelberg M. Jenkins Gordon Morison David McMinn Z. Tieges 27 6 0 24 Mar 2023
On the Blind Spots of Model-Based Evaluation Metrics for Text Generation Tianxing He Jingyu Zhang Tianle Wang Sachin Kumar Kyunghyun Cho James R. Glass Yulia Tsvetkov 40 44 0 20 Dec 2022
BUMP: A Benchmark of Unfaithful Minimal Pairs for Meta-Evaluation of Faithfulness Metrics Liang Ma Shuyang Cao IV RobertL.Logan Di Lu Shihao Ran Kecheng Zhang Joel R. Tetreault A. Jaimes 17 6 0 20 Dec 2022
Improving Faithfulness of Abstractive Summarization by Controlling Confounding Effect of Irrelevant Sentences Asish Ghoshal Arash Einolghozati A. Arun Haoran Li L. Yu Vera Gor Yashar Mehdad Scott Yih Asli Celikyilmaz HILM 21 1 0 19 Dec 2022
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation Yixin Liu Alexander R. Fabbri Pengfei Liu Yilun Zhao Linyong Nan ... Simeng Han Shafiq R. Joty Chien-Sheng Wu Caiming Xiong Dragomir R. Radev ALM 12 132 0 15 Dec 2022
NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries? Saadia Gabriel Hamid Palangi Yejin Choi AAML 37 1 0 08 Nov 2022
How Far are We from Robust Long Abstractive Summarization? Huan Yee Koh Jiaxin Ju He Zhang Ming Liu Shirui Pan HILM 23 39 0 30 Oct 2022
Just ClozE! A Novel Framework for Evaluating the Factual Consistency Faster in Abstractive Summarization Yiyang Li Lei Li Marina Litvak N. Vanetik Dingxing Hu Yuze Li Yanquan Zhou HILM 38 0 0 06 Oct 2022
An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics Huan Yee Koh Jiaxin Ju Ming Liu Shirui Pan 75 122 0 03 Jul 2022
Conditional Generation with a Question-Answering Blueprint Shashi Narayan Joshua Maynez Reinald Kim Amplayo Kuzman Ganchev Annie Louis Fantine Huot Anders Sandholm Dipanjan Das Mirella Lapata 54 47 0 01 Jul 2022
QRelScore: Better Evaluating Generated Questions with Deeper Understanding of Context-aware Relevance Xiaoqiang Wang Bang Liu Siliang Tang Lingfei Wu 30 9 0 29 Apr 2022
Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarization Yue Dong John Wieting Pat Verga HILM 21 24 0 28 Apr 2022
A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generation Shashi Narayan Gonccalo Simoes Yao-Min Zhao Joshua Maynez Dipanjan Das Michael Collins Mirella Lapata 29 30 0 28 Mar 2022
Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processing Tal August Lucy Lu Wang Jonathan Bragg Marti A. Hearst Andrew Head Kyle Lo 15 40 0 28 Feb 2022
Survey of Hallucination in Natural Language Generation Ziwei Ji Nayeon Lee Rita Frieske Tiezheng Yu D. Su ... Delong Chen Wenliang Dai Ho Shu Chan Andrea Madotto Pascale Fung HILM LRM 49 2,234 0 08 Feb 2022
Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries Xiangru Tang Alexander R. Fabbri Haoran Li Ziming Mao Griffin Adams Borui Wang Asli Celikyilmaz Yashar Mehdad Dragomir R. Radev HILM 13 19 0 19 Sep 2021
ComSum: Commit Messages Summarization and Meaning Preservation Leshem Choshen Idan Amit 17 4 0 23 Aug 2021
A Survey on Dialogue Summarization: Recent Advances and New Frontiers Xiachong Feng Xiaocheng Feng Bing Qin 32 100 0 07 Jul 2021
Towards Human-Free Automatic Quality Evaluation of German Summarization Neslihan Iskender Oleg V. Vasilyev Tim Polzehl John Bohannon Sebastian Möller 21 1 0 13 May 2021
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark Nouha Dziri Hannah Rashkin Tal Linzen David Reitter ALM 189 79 0 30 Apr 2021
The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey Yi-Chong Huang Xiachong Feng Xiaocheng Feng Bing Qin HILM 133 104 0 30 Apr 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics Sebastian Gehrmann Tosin P. Adewumi Karmanya Aggarwal Pawan Sasanka Ammanamanchi Aremu Anuoluwapo ... Nishant Subramani Wei-ping Xu Diyi Yang Akhila Yerukola Jiawei Zhou VLM 248 285 0 02 Feb 2021
Enhancing Factual Consistency of Abstractive Summarization Chenguang Zhu William Fu-Hinthorn Ruochen Xu Qingkai Zeng Michael Zeng Xuedong Huang Meng-Long Jiang HILM KELM 190 40 0 19 Mar 2020