QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization

16 December 2021

Papers citing "QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization"

50 / 167 papers shown

Title
Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems Lidiya Murakhovs'ka Philippe Laban Tian Xie Caiming Xiong Chien-Sheng Wu 38 6 0 26 Oct 2023
Fast and Accurate Factual Inconsistency Detection Over Long Documents B. Lattimer Patrick Chen Xinyuan Zhang Yi Yang HILM 13 18 0 19 Oct 2023
Visual Storytelling with Question-Answer Plans Danyang Liu Mirella Lapata Frank Keller CoGe 24 9 0 08 Oct 2023
Beyond the Chat: Executable and Verifiable Text-Editing with LLMs Philippe Laban Jesse Vig Marti A. Hearst Caiming Xiong Chien-Sheng Wu KELM 37 27 0 27 Sep 2023
Large Language Model Alignment: A Survey Tianhao Shen Renren Jin Yufei Huang Chuang Liu Weilong Dong Zishan Guo Xinwei Wu Yan Liu Deyi Xiong LM&MA 29 179 0 26 Sep 2023
ExpertQA: Expert-Curated Questions and Attributed Answers Chaitanya Malaviya Subin Lee Sihao Chen Elizabeth Sieber Mark Yatskar Dan Roth ELM HILM 38 52 0 14 Sep 2023
FaNS: a Facet-based Narrative Similarity Metric Mousumi Akter Shubhra (Santu) Karmaker 30 1 0 09 Sep 2023
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models Yue Zhang Yafu Li Leyang Cui Deng Cai Lemao Liu ... Longyue Wang Anh Tuan Luu Wei Bi Freda Shi Shuming Shi RALM LRM HILM 53 523 0 03 Sep 2023
Automatically Evaluating Opinion Prevalence in Opinion Summarization Christopher Malon 17 1 0 26 Jul 2023
FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios Ethan Chern Steffi Chern Shiqi Chen Weizhe Yuan Kehua Feng Chunting Zhou Junxian He Graham Neubig Pengfei Liu HILM 32 194 0 25 Jul 2023
Generating Benchmarks for Factuality Evaluation of Language Models Dor Muhlgay Ori Ram Inbal Magar Yoav Levine Nir Ratner Yonatan Belinkov Omri Abend Kevin Leyton-Brown Amnon Shashua Y. Shoham HILM 38 91 0 13 Jul 2023
Improving Factuality of Abstractive Summarization via Contrastive Reward Learning Ethan Chern Zhiruo Wang Sanjan Das Bhavuk Sharma Pengfei Liu Graham Neubig HILM 20 14 0 10 Jul 2023
Text Alignment Is An Efficient Unified Model for Massive NLP Tasks Yuheng Zha Yichi Yang Ruichen Li Zhiting Hu ALM 24 9 0 06 Jul 2023
Conformal Language Modeling Victor Quach Adam Fisch Tal Schuster Adam Yala J. Sohn Tommi Jaakkola Regina Barzilay 85 56 0 16 Jun 2023
Reference Matters: Benchmarking Factual Error Correction for Dialogue Summarization with Fine-grained Evaluation Framework Mingqi Gao Xiaojun Wan Jia Su Zhefeng Wang Baoxing Huai HILM 21 8 0 08 Jun 2023
Concise Answers to Complex Questions: Summarization of Long-form Answers Abhilash Potluri Fangyuan Xu Eunsol Choi ELM 39 11 0 30 May 2023
A Critical Evaluation of Evaluations for Long-form Question Answering Fangyuan Xu Yixiao Song Mohit Iyyer Eunsol Choi ELM 44 97 0 29 May 2023
AlignScore: Evaluating Factual Consistency with a Unified Alignment Function Yuheng Zha Yichi Yang Ruichen Li Zhiting Hu HILM 26 182 0 26 May 2023
Annotating and Detecting Fine-grained Factual Errors for Dialogue Summarization Rongxin Zhu Jianzhong Qi Jey Han Lau 54 10 0 26 May 2023
AWESOME: GPU Memory-constrained Long Document Summarization using Memory Mechanism and Global Salient Content Shuyang Cao Lu Wang 30 5 0 24 May 2023
DecipherPref: Analyzing Influential Factors in Human Preference Judgments via GPT-4 Ye Hu Kaiqiang Song Sangwoo Cho Xiaoyang Wang H. Foroosh Fei Liu 31 12 0 24 May 2023
Interpretable Automatic Fine-grained Inconsistency Detection in Text Summarization Hou Pong Chan Qi Zeng Chenhui Xu HILM 42 12 0 23 May 2023
LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond Philippe Laban Wojciech Kry'sciñski Divyansh Agarwal Alexander R. Fabbri Caiming Xiong Shafiq Joty Chien-Sheng Wu ALM HILM 38 33 0 23 May 2023
USB: A Unified Summarization Benchmark Across Tasks and Domains Kundan Krishna Prakhar Gupta S. Ramprasad Byron C. Wallace Jeffrey P. Bigham Zachary Chase Lipton HILM 48 8 0 23 May 2023
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation Sewon Min Kalpesh Krishna Xinxi Lyu M. Lewis Wen-tau Yih Pang Wei Koh Mohit Iyyer Luke Zettlemoyer Hannaneh Hajishirzi HILM ALM 88 611 0 23 May 2023
ManiTweet: A New Benchmark for Identifying Manipulation of News on Social Media Kung-Hsiang Huang Hou Pong Chan Kathleen McKeown Heng Ji 41 1 0 23 May 2023
Evaluating Factual Consistency of Summaries with Large Language Models Shiqi Chen Siyang Gao Junxian He ELM LRM HILM 37 6 0 23 May 2023
Detecting and Mitigating Hallucinations in Multilingual Summarisation Yifu Qiu Yftah Ziser Anna Korhonen Edoardo Ponti Shay B. Cohen HILM 61 43 0 23 May 2023
Attributable and Scalable Opinion Summarization Tom Hosking Hao Tang Mirella Lapata 36 8 0 19 May 2023
Counterfactual Debiasing for Generating Factually Consistent Text Summaries Chenhe Dong Yuexiang Xie Yaliang Li Ying Shen CML HILM 36 0 0 18 May 2023
FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge Shangbin Feng Vidhisha Balachandran Yuyang Bai Yulia Tsvetkov KELM HILM 34 52 0 14 May 2023
Zero-shot Faithful Factual Error Correction Kung-Hsiang Huang Hou Pong Chan Heng Ji KELM HILM 37 30 0 13 May 2023
Expository Text Generation: Imitate, Retrieve, Paraphrase Nishant Balepur Jie Huang Kevin Chen-Chuan Chang 25 8 0 05 May 2023
Personalized Abstractive Summarization by Tri-agent Generation Pipeline Md Aminul Haque Palash Sourav Saha Faria Afrin Pengcheng He 47 4 0 04 May 2023
Can LMs Generalize to Future Data? An Empirical Analysis on Text Summarization C. Cheang Hou Pong Chan Derek F. Wong Xuebo Liu Zhao Li Yanming Sun Shudong Liu Lidia S. Chao 205 6 0 03 May 2023
A Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarization Griffin Adams Jason Zucker Noémie Elhadad 57 23 0 07 Mar 2023
Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation Yixin Liu Alexander R. Fabbri Yilun Zhao Pengfei Liu Shafiq Joty Chien-Sheng Wu Caiming Xiong Dragomir R. Radev 15 28 0 07 Mar 2023
Benchmarking Large Language Models for News Summarization Tianyi Zhang Faisal Ladhak Esin Durmus Percy Liang Kathleen McKeown Tatsunori B. Hashimoto ELM 43 487 0 31 Jan 2023
LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization Kalpesh Krishna Erin Bransom Bailey Kuehl Mohit Iyyer Pradeep Dasigi Arman Cohan Kyle Lo 24 90 0 30 Jan 2023
SWING: Balancing Coverage and Faithfulness for Dialogue Summarization Kung-Hsiang Huang Siffi Singh Xiaofei Ma Wei Xiao Wei Xiao Nicholas Dingwall William Yang Wang Kathleen McKeown HILM 40 13 0 25 Jan 2023
Rethinking with Retrieval: Faithful Large Language Model Inference Hangfeng He Hongming Zhang Dan Roth KELM LRM 149 161 0 31 Dec 2022
mFACE: Multilingual Summarization with Factual Consistency Evaluation Roee Aharoni Shashi Narayan Joshua Maynez Jonathan Herzig Elizabeth Clark Mirella Lapata HILM 29 44 0 20 Dec 2022
WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning Wenhao Wu Wei Li Xinyan Xiao Jiachen Liu Sujian Li Yajuan Lv HILM 31 4 0 20 Dec 2022
On Improving Summarization Factual Consistency from Natural Language Feedback Yixin Liu Budhaditya Deb Milagro Teruel Aaron L Halfaker Dragomir R. Radev Ahmed Hassan Awadallah HILM 31 35 0 20 Dec 2022
BUMP: A Benchmark of Unfaithful Minimal Pairs for Meta-Evaluation of Faithfulness Metrics Liang Ma Shuyang Cao IV RobertL.Logan Di Lu Shihao Ran Kecheng Zhang Joel R. Tetreault A. Jaimes 17 6 0 20 Dec 2022
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation Yixin Liu Alexander R. Fabbri Pengfei Liu Yilun Zhao Linyong Nan ... Simeng Han Shafiq Joty Chien-Sheng Wu Caiming Xiong Dragomir R. Radev ALM 31 133 0 15 Dec 2022
Improved Beam Search for Hallucination Mitigation in Abstractive Summarization A. Sridhar Erik M. Visser HILM 30 14 0 06 Dec 2022
Best- $k$ Search Algorithm for Neural Text Generation Jiacheng Xu Caiming Xiong Silvio Savarese Yingbo Zhou 37 5 0 22 Nov 2022
Improving Factual Consistency in Summarization with Compression-Based Post-Editing Alexander R. Fabbri Prafulla Kumar Choubey Jesse Vig Chien-Sheng Wu Caiming Xiong HILM KELM 49 17 0 11 Nov 2022
Discord Questions: A Computational Approach To Diversity Analysis in News Coverage Philippe Laban Chien-Sheng Wu Lidiya Murakhovs'ka Xiang Ánthony' Chen Caiming Xiong 35 12 0 09 Nov 2022