QuestEval: Summarization Asks for Fact-based Evaluation

23 March 2021

Papers citing "QuestEval: Summarization Asks for Fact-based Evaluation"

50 / 181 papers shown

Title
Interpretable Automatic Fine-grained Inconsistency Detection in Text Summarization Hou Pong Chan Qi Zeng Chenhui Xu HILM 29 12 0 23 May 2023
LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond Philippe Laban Wojciech Kry'sciñski Divyansh Agarwal Alexander R. Fabbri Caiming Xiong Shafiq Joty Chien-Sheng Wu ALM HILM 35 33 0 23 May 2023
USB: A Unified Summarization Benchmark Across Tasks and Domains Kundan Krishna Prakhar Gupta S. Ramprasad Byron C. Wallace Jeffrey P. Bigham Zachary Chase Lipton HILM 43 8 0 23 May 2023
Evaluating Factual Consistency of Summaries with Large Language Models Shiqi Chen Siyang Gao Junxian He ELM LRM HILM 37 6 0 23 May 2023
Detecting and Mitigating Hallucinations in Multilingual Summarisation Yifu Qiu Yftah Ziser Anna Korhonen Edoardo Ponti Shay B. Cohen HILM 61 43 0 23 May 2023
LM vs LM: Detecting Factual Errors via Cross Examination Roi Cohen May Hamri Mor Geva Amir Globerson HILM 41 120 0 22 May 2023
TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models Zorik Gekhman Jonathan Herzig Roee Aharoni Chen Elkind Idan Szpektor HILM ELM 31 72 0 18 May 2023
What You See is What You Read? Improving Text-Image Alignment Evaluation Michal Yarom Yonatan Bitton Soravit Changpinyo Roee Aharoni Jonathan Herzig Oran Lang E. Ofek Idan Szpektor EGVM 59 74 0 17 May 2023
FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge Shangbin Feng Vidhisha Balachandran Yuyang Bai Yulia Tsvetkov KELM HILM 29 52 0 14 May 2023
Expository Text Generation: Imitate, Retrieve, Paraphrase Nishant Balepur Jie Huang Kevin Chen-Chuan Chang 16 8 0 05 May 2023
Extractive Summarization via ChatGPT for Faithful Summary Generation Haopeng Zhang Xiao Liu Jiawei Zhang 38 76 0 09 Apr 2023
ChatGPT as a Factual Inconsistency Evaluator for Text Summarization Zheheng Luo Qianqian Xie Sophia Ananiadou ELM HILM ALM 41 74 0 27 Mar 2023
Faithfulness-Aware Decoding Strategies for Abstractive Summarization David Wan Mengwen Liu Kathleen McKeown Markus Dreyer Joey Tianyi Zhou HILM 111 32 0 06 Mar 2023
WiCE: Real-World Entailment for Claims in Wikipedia Ryo Kamoi Tanya Goyal Juan Diego Rodriguez Greg Durrett 41 81 0 02 Mar 2023
Complex QA and language models hybrid architectures, Survey Xavier Daull P. Bellot Emmanuel Bruno Vincent Martin Elisabeth Murisasco ELM 31 15 0 17 Feb 2023
GPTScore: Evaluate as You Desire Jinlan Fu See-Kiong Ng Zhengbao Jiang Pengfei Liu LM&MA ALM ELM 15 268 0 08 Feb 2023
Leveraging Summary Guidance on Medical Report Summarization Yunqi Zhu Xuebing Yang Yuanyuan Wu Wensheng Zhang 33 9 0 08 Feb 2023
Do Multi-Document Summarization Models Synthesize? Jay DeYoung Stephanie C. Martinez Iain J. Marshall Byron C. Wallace 24 8 0 31 Jan 2023
Improving Open-Domain Dialogue Evaluation with a Causal Inference Model Cat P. Le Luke Dai Michael Johnston Yang Liu M. Walker R. Ghanadan ELM 19 10 0 31 Jan 2023
MQAG: Multiple-choice Question Answering and Generation for Assessing Information Consistency in Summarization Potsawee Manakul Adian Liusie Mark Gales HILM 21 35 0 28 Jan 2023
mFACE: Multilingual Summarization with Factual Consistency Evaluation Roee Aharoni Shashi Narayan Joshua Maynez Jonathan Herzig Elizabeth Clark Mirella Lapata HILM 27 44 0 20 Dec 2022
WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning Wenhao Wu Wei Li Xinyan Xiao Jiachen Liu Sujian Li Yajuan Lv HILM 31 4 0 20 Dec 2022
BUMP: A Benchmark of Unfaithful Minimal Pairs for Meta-Evaluation of Faithfulness Metrics Liang Ma Shuyang Cao IV RobertL.Logan Di Lu Shihao Ran Kecheng Zhang Joel R. Tetreault A. Jaimes 17 6 0 20 Dec 2022
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation Yixin Liu Alexander R. Fabbri Pengfei Liu Yilun Zhao Linyong Nan ... Simeng Han Shafiq Joty Chien-Sheng Wu Caiming Xiong Dragomir R. Radev ALM 31 133 0 15 Dec 2022
Scientific Paper Extractive Summarization Enhanced by Citation Graphs Preslav Nakov Mingzhe Li Shen Gao Rui Yan Xin Gao Xiangliang Zhang 25 12 0 08 Dec 2022
RHO ( $ρ$ ): Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding Ziwei Ji Zihan Liu Nayeon Lee Tiezheng Yu Bryan Wilie Mini Zeng Pascale Fung HILM 23 53 0 03 Dec 2022
Prompted Opinion Summarization with GPT-3.5 Adithya Bhaskar Alexander R. Fabbri Greg Durrett ELM 19 51 0 29 Nov 2022
HaRiM $^+$ : Evaluating Summary Quality with Hallucination Risk Seonil Son Junsoo Park J. Hwang Junghwa Lee Hyungjong Noh Yeonsoo Lee HILM 19 8 0 22 Nov 2022
Evaluating the Factual Consistency of Large Language Models Through News Summarization Derek Tam Anisha Mascarenhas Shiyue Zhang Sarah Kwan Joey Tianyi Zhou Colin Raffel HILM 33 96 0 15 Nov 2022
Evaluating and Improving Factuality in Multimodal Abstractive Summarization David Wan Joey Tianyi Zhou 20 10 0 04 Nov 2022
Time-aware Prompting for Text Generation Shuyang Cao Lu Wang 29 11 0 03 Nov 2022
RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question Alireza Mohammadshahi Thomas Scialom Majid Yazdani Pouya Yanki Angela Fan James Henderson Marzieh Saeidi 31 20 0 02 Nov 2022
Improving abstractive summarization with energy-based re-ranking Diogo Pernes Afonso Mendes André F. T. Martins 23 6 0 27 Oct 2022
How "Multi" is Multi-Document Summarization? Ruben Wolhandler Arie Cattan Ori Ernst Ido Dagan 84 12 0 23 Oct 2022
On the Limitations of Reference-Free Evaluations of Generated Text Daniel Deutsch Rotem Dror Dan Roth 40 45 0 22 Oct 2022
Precisely the Point: Adversarial Augmentations for Faithful and Informative Text Generation Wenhao Wu Wei Li Jiachen Liu Xinyan Xiao Sujian Li Yajuan Lyu 42 3 0 22 Oct 2022
Analyzing and Evaluating Faithfulness in Dialogue Summarization Bin Wang Chen Zhang Yan Zhang Yiming Chen Haizhou Li HILM 41 15 0 21 Oct 2022
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey Sachin Kumar Vidhisha Balachandran Lucille Njoo Antonios Anastasopoulos Yulia Tsvetkov ELM 81 85 0 14 Oct 2022
Towards a Unified Multi-Dimensional Evaluator for Text Generation Ming Zhong Yang Liu Da Yin Yuning Mao Yizhu Jiao Peng Liu Chenguang Zhu Heng Ji Jiawei Han ELM 50 256 0 13 Oct 2022
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization Ryo Kamoi Tanya Goyal Greg Durrett HILM 44 14 0 13 Oct 2022
Just ClozE! A Novel Framework for Evaluating the Factual Consistency Faster in Abstractive Summarization Yiyang Li Lei Li Marina Litvak N. Vanetik Dingxing Hu Yuze Li Yanquan Zhou HILM 40 0 0 06 Oct 2022
Towards Improving Faithfulness in Abstractive Summarization Preslav Nakov Mingzhe Li Xin Gao Xiangliang Zhang HILM 33 26 0 04 Oct 2022
News Summarization and Evaluation in the Era of GPT-3 Tanya Goyal Junyi Jessy Li Greg Durrett ELM 31 387 0 26 Sep 2022
Summarization Programs: Interpretable Abstractive Summarization with Neural Modular Trees Swarnadeep Saha Shiyue Zhang Peter Hase Joey Tianyi Zhou 29 19 0 21 Sep 2022
Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarization Shiyue Zhang David Wan Joey Tianyi Zhou HILM 52 27 0 08 Sep 2022
Podcast Summary Assessment: A Resource for Evaluating Summary Assessment Methods Potsawee Manakul Mark Gales 15 5 0 28 Aug 2022
PEER: A Collaborative Language Model Timo Schick Jane Dwivedi-Yu Zhengbao Jiang Fabio Petroni Patrick Lewis Gautier Izacard Qingfei You Christoforos Nalmpantis Edouard Grave Sebastian Riedel ALM 50 93 0 24 Aug 2022
Joint Generator-Ranker Learning for Natural Language Generation Weizhou Shen Yeyun Gong Yelong Shen Song Wang Xiaojun Quan Nan Duan Weizhu Chen 42 5 0 28 Jun 2022
EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RL Thomas Carta Pierre-Yves Oudeyer Olivier Sigaud Sylvain Lamprier OffRL 28 24 0 20 Jun 2022
Counseling Summarization using Mental Health Knowledge Guided Utterance Filtering Aseem Srivastava Tharun Suresh Sarah Peregrine S. P. Lord Md. Shad Akhtar Tanmoy Chakraborty 19 13 0 08 Jun 2022