Question Answering as an Automatic Evaluation Metric for News Article Summarization

2 June 2019

Papers citing "Question Answering as an Automatic Evaluation Metric for News Article Summarization"

38 / 38 papers shown

Title
SteLLA: A Structured Grading System Using LLMs with RAG Hefei Qiu Brian White Ashley Ding Reinaldo Costa Ali Hachem Wei Ding Ping Chen AI4Ed 61 0 0 17 Jan 2025
ROUGE-K: Do Your Summaries Have Keywords? Sotaro Takeshita Simone Paolo Ponzetto Kai Eckert 22 0 0 08 Mar 2024
Discharge Summary Hospital Course Summarisation of In Patient Electronic Health Record Text with Clinical Concept Guided Deep Pre-Trained Transformer Models Thomas Searle Zina M. Ibrahim J. Teo Richard J. B. Dobson 21 29 0 14 Nov 2022
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization Ryo Kamoi Tanya Goyal Greg Durrett HILM 44 14 0 13 Oct 2022
An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics Huan Yee Koh Jiaxin Ju Ming Liu Shirui Pan 83 122 0 03 Jul 2022
Conditional Generation with a Question-Answering Blueprint Shashi Narayan Joshua Maynez Reinald Kim Amplayo Kuzman Ganchev Annie Louis Fantine Huot Anders Sandholm Dipanjan Das Mirella Lapata 61 47 0 01 Jul 2022
QASem Parsing: Text-to-text Modeling of QA-based Semantics Ayal Klein Eran Hirsch Ron Eliav Valentina Pyatkin Avi Caciularu Ido Dagan 49 12 0 23 May 2022
Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarization Prasetya Ajie Utama Joshua Bambrick N. Moosavi Iryna Gurevych HILM 18 42 0 12 May 2022
Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters Tal Schuster Sihao Chen S. Buthpitiya Alex Fabrikant Donald Metzler 26 41 0 15 Apr 2022
Evaluation of Automatic Text Summarization using Synthetic Facts J. Ahn Foaad Khosmood HILM 18 0 0 11 Apr 2022
Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluation Jannis Bulian Christian Buck Wojciech Gajewski Benjamin Boerschinger Tal Schuster 31 44 0 15 Feb 2022
QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization Alexander R. Fabbri C. Wu Wenhao Liu Caiming Xiong HILM 22 208 0 16 Dec 2021
Cut the CARP: Fishing for zero-shot story evaluation Shahbuland Matiana J. Smith Ryan Teehan Louis Castricato Stella Biderman Leo Gao Spencer Frazier 49 16 0 06 Oct 2021
Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries Xiangru Tang Alexander R. Fabbri Haoran Li Ziming Mao Griffin Adams Borui Wang Asli Celikyilmaz Yashar Mehdad Dragomir R. Radev HILM 13 19 0 19 Sep 2021
Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generation Mingkai Deng Bowen Tan Zhengzhong Liu Eric Xing Zhiting Hu 16 73 0 14 Sep 2021
Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation Yuexiang Xie Fei Sun Yang Deng Yaliang Li Bolin Ding HILM 26 53 0 30 Aug 2021
RetGen: A Joint framework for Retrieval and Grounded Text Generation Modeling Yizhe Zhang Siqi Sun Xiang Gao Yuwei Fang Chris Brockett Michel Galley Jianfeng Gao Bill Dolan RALM 38 30 0 14 May 2021
Towards Human-Free Automatic Quality Evaluation of German Summarization Neslihan Iskender Oleg V. Vasilyev Tim Polzehl John Bohannon Sebastian Möller 29 1 0 13 May 2021
The Summary Loop: Learning to Write Abstractive Summaries Without Examples Philippe Laban Andrew Hsi Bloomberg John F. Canny Marti A. Hearst 25 56 0 11 May 2021
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation Tianyu Liu Yizhe Zhang Chris Brockett Yi Mao Zhifang Sui Weizhu Chen W. Dolan HILM 228 144 0 18 Apr 2021
$$Q^{2}$: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering$ $Q^{2}$ : Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering Or Honovich Leshem Choshen Roee Aharoni Ella Neeman Idan Szpektor Omri Abend HILM 36 138 0 16 Apr 2021
What's in a Summary? Laying the Groundwork for Advances in Hospital-Course Summarization Griffin Adams Emily Alsentzer Mert Ketenci Jason Zucker Noémie Elhadad 52 47 0 12 Apr 2021
QuestEval: Summarization Asks for Fact-based Evaluation Thomas Scialom Paul-Alexis Dray Patrick Gallinari Sylvain Lamprier Benjamin Piwowarski Jacopo Staiano Alex Jinpeng Wang HILM 19 267 0 23 Mar 2021
Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View Tianyu Liu Xin Zheng Baobao Chang Zhifang Sui 127 35 0 17 Feb 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics Sebastian Gehrmann Tosin P. Adewumi Karmanya Aggarwal Pawan Sasanka Ammanamanchi Aremu Anuoluwapo ... Nishant Subramani Wei-ping Xu Diyi Yang Akhila Yerukola Jiawei Zhou VLM 260 285 0 02 Feb 2021
CapWAP: Captioning with a Purpose Adam Fisch Kenton Lee Ming-Wei Chang J. Clark Regina Barzilay 8 11 0 09 Nov 2020
Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary Daniel Deutsch Tania Bedrax-Weiss Dan Roth 8 109 0 01 Oct 2020
Evaluation of Text Generation: A Survey Asli Celikyilmaz Elizabeth Clark Jianfeng Gao ELM LM&MA 19 377 0 26 Jun 2020
FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization Esin Durmus He He Mona T. Diab HILM 23 384 0 07 May 2020
Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward Luyang Huang Lingfei Wu Lu Wang RALM 32 161 0 03 May 2020
MLSUM: The Multilingual Summarization Corpus Thomas Scialom Paul-Alexis Dray Sylvain Lamprier Benjamin Piwowarski Jacopo Staiano 32 173 0 30 Apr 2020
BLEURT: Learning Robust Metrics for Text Generation Thibault Sellam Dipanjan Das Ankur P. Parikh 46 1,450 0 09 Apr 2020
Asking and Answering Questions to Evaluate the Factual Consistency of Summaries Alex Jinpeng Wang Kyunghyun Cho M. Lewis HILM 36 472 0 08 Apr 2020
Fill in the BLANC: Human-free quality estimation of document summaries Oleg V. Vasilyev Vedant Dharnidharka John Bohannon 3DH 42 116 0 23 Feb 2020
Answers Unite! Unsupervised Metrics for Reinforced Summarization Models Thomas Scialom Sylvain Lamprier Benjamin Piwowarski Jacopo Staiano 27 149 0 04 Sep 2019
OpenNMT: Open-Source Toolkit for Neural Machine Translation Guillaume Klein Yoon Kim Yuntian Deng Jean Senellart Alexander M. Rush 273 1,896 0 10 Jan 2017
SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents Ramesh Nallapati Feifei Zhai Bowen Zhou 207 1,255 0 14 Nov 2016
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 718 6,748 0 26 Sep 2016