Evaluating the Factual Consistency of Abstractive Text Summarization

28 October 2019

Papers citing "Evaluating the Factual Consistency of Abstractive Text Summarization"

50 / 463 papers shown

Title
FineSurE: Fine-grained Summarization Evaluation using LLMs Hwanjun Song Hang Su Igor Shalyminov Jason (Jinglun) Cai Saab Mansour HILM 41 32 0 01 Jul 2024
Detection and Measurement of Syntactic Templates in Generated Text Chantal Shaib Yanai Elazar Junyi Jessy Li Byron C. Wallace 54 16 0 28 Jun 2024
Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics Weijia Zhang Mohammad Aliannejadi Yifei Yuan Jiahuan Pei Jia-Hong Huang Evangelos Kanoulas HILM 31 12 0 21 Jun 2024
Factual Dialogue Summarization via Learning from Large Language Models Rongxin Zhu Jey Han Lau Jianzhong Qi HILM 55 1 0 20 Jun 2024
FoRAG: Factuality-optimized Retrieval Augmented Generation for Web-enhanced Long-form Question Answering Tianchi Cai Zhiwen Tan Xierui Song Tao Sun Jiyan Jiang Yunqi Xu Yinger Zhang Jinjie Gu 32 5 0 19 Jun 2024
Learning to Generate Answers with Citations via Factual Consistency Models Rami Aly Zhiqiang Tang Samson Tan George Karypis HILM 42 5 0 19 Jun 2024
Detecting Errors through Ensembling Prompts (DEEP): An End-to-End LLM Framework for Detecting Factual Errors Alex Chandler Devesh Surve Hui Su HILM UQCV 31 1 0 18 Jun 2024
EMO-KNOW: A Large Scale Dataset on Emotion and Emotion-cause M. Nguyen Yasith Samaradivakara P. Sasikumar Chitralekha Gupta Suranga Nanayakkara 42 1 0 18 Jun 2024
A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models Haopeng Zhang Philip S. Yu Jiawei Zhang 37 17 0 17 Jun 2024
Mitigating Large Language Model Hallucination with Faithful Finetuning Minda Hu Bowei He Yufei Wang Liangyou Li Chen Ma Irwin King HILM 46 7 0 17 Jun 2024
GLIMPSE: Pragmatically Informative Multi-Document Summarization for Scholarly Reviews Maxime Darrin Ines Arous Pablo Piantanida Jackie CK Cheung 55 2 0 11 Jun 2024
Key-Element-Informed sLLM Tuning for Document Summarization Sangwon Ryu Heejin Do Yunsu Kim G. G. Lee Jungseul Ok 37 6 0 07 Jun 2024
PatentEval: Understanding Errors in Patent Generation You Zuo Kim Gerdes Eric Villemonte de la Clergerie Benoît Sagot 34 1 0 05 Jun 2024
Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework Xiaoxi Sun Jinpeng Li Yan Zhong Dongyan Zhao Rui Yan LLMAG HILM 29 5 0 05 Jun 2024
TruthEval: A Dataset to Evaluate LLM Truthfulness and Reliability Aisha Khatun Daniel G. Brown HILM 29 2 0 04 Jun 2024
Facilitating Human-LLM Collaboration through Factuality Scores and Source Attributions Hyo Jin Do Rachel Ostrand Justin D. Weisz Casey Dugan P. Sattigeri Dennis L. Wei K. Murugesan Werner Geyer HILM 45 10 0 30 May 2024
Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? Zorik Gekhman G. Yona Roee Aharoni Matan Eyal Amir Feder Roi Reichart Jonathan Herzig 52 109 0 09 May 2024
Efficient LLM Comparative Assessment: a Product of Experts Framework for Pairwise Comparisons Adian Liusie Vatsal Raina Yassir Fathullah Mark Gales 43 10 0 09 May 2024
QANA: LLM-based Question Generation and Network Analysis for Zero-shot Key Point Analysis and Beyond Tomoki Fukuma Koki Noda Toshihide Ubukata Kousuke Hoso Yoshiharu Ichikawa Kyosuke Kambe Yu Masubuchi F. Toriumi 32 0 0 29 Apr 2024
Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratings Olivia Wiles Chuhan Zhang Isabela Albuquerque Ivana Kajić Su Wang ... Jordi Pont-Tuset Aida Nematzadeh Anant Nawalgaria Jordi Pont-Tuset Aida Nematzadeh EGVM 135 14 0 25 Apr 2024
Can We Catch the Elephant? A Survey of the Evolvement of Hallucination Evaluation on Natural Language Generation Siya Qi Yulan He Zheng Yuan LRM HILM 49 1 0 18 Apr 2024
FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document Joonho Yang Seunghyun Yoon Byeongjeong Kim Hwanhee Lee HILM 34 5 0 17 Apr 2024
MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents Liyan Tang Philippe Laban Greg Durrett HILM SyDa 43 78 0 16 Apr 2024
Benchmarking Llama2, Mistral, Gemma and GPT for Factuality, Toxicity, Bias and Propensity for Hallucinations David Nadeau Mike Kroutikov Karen McNeil Simon Baribeau HILM 29 7 0 15 Apr 2024
Mitigating Hallucination in Abstractive Summarization with Domain-Conditional Mutual Information Kyubyung Chae Jaepill Choi Yohan Jo Taesup Kim HILM 30 1 0 15 Apr 2024
WikiSplit++: Easy Data Refinement for Split and Rephrase Hayato Tsukagoshi Tsutomu Hirao Makoto Morishita Katsuki Chousa Ryohei Sasano Koichi Takeda 43 1 0 13 Apr 2024
The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Models Giwon Hong Aryo Pradipta Gema Rohit Saxena Xiaotang Du Ping Nie ... Laura Perez-Beltrachini Max Ryabinin Xuanli He Clémentine Fourrier Pasquale Minervini LRM HILM 38 11 0 08 Apr 2024
Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators Yann Dubois Balázs Galambosi Percy Liang Tatsunori Hashimoto ALM 55 330 0 06 Apr 2024
Schroedinger's Threshold: When the AUC doesn't predict Accuracy Juri Opitz UQCV 41 0 0 04 Apr 2024
Evaluating Document Simplification: On the Importance of Separately Assessing Simplicity and Meaning Preservation Liam Cripwell Joël Legrand Claire Gardent 31 3 0 04 Apr 2024
ALOHa: A New Measure for Hallucination in Captioning Models Suzanne Petryk David M. Chan Anish Kachinthaya Haodi Zou John F. Canny Joseph E. Gonzalez Trevor Darrell HILM 42 11 0 03 Apr 2024
Multi-Review Fusion-in-Context Aviv Slobodkin Ori Shapira Ran Levy Ido Dagan 143 1 0 22 Mar 2024
SIFiD: Reassess Summary Factual Inconsistency Detection with LLM Jiuding Yang Hui Liu Weidong Guo Zhuwei Rao Yu-Syuan Xu Di Niu HILM 23 0 0 12 Mar 2024
ROUGE-K: Do Your Summaries Have Keywords? Sotaro Takeshita Simone Paolo Ponzetto Kai Eckert 22 0 0 08 Mar 2024
Semi-Supervised Dialogue Abstractive Summarization via High-Quality Pseudolabel Selection Jianfeng He Hang Su Jason (Jinglun) Cai Igor Shalyminov Hwanjun Song Saab Mansour 32 4 0 06 Mar 2024
A Modular Approach for Multimodal Summarization of TV Shows Louis Mahon Mirella Lapata 29 10 0 06 Mar 2024
In Search of Truth: An Interrogation Approach to Hallucination Detection Yakir Yehuda Itzik Malkiel Oren Barkan Jonathan Weill Royi Ronen Noam Koenigstein HILM 29 8 0 05 Mar 2024
A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods Hanlei Jin Yang Zhang Dan Meng Jun Wang Jinghua Tan 68 80 0 05 Mar 2024
FENICE: Factuality Evaluation of summarization based on Natural language Inference and Claim Extraction Alessandro Sciré Karim Ghonim Roberto Navigli HILM 29 8 0 04 Mar 2024
Self-Consistent Decoding for More Factual Open Responses Christopher Malon Xiaodan Zhu HILM 46 3 0 01 Mar 2024
Reducing Hallucinations in Entity Abstract Summarization with Facts-Template Decomposition Fangwei Zhu Peiyi Wang Zhifang Sui HILM 37 2 0 29 Feb 2024
Fine-Grained Natural Language Inference Based Faithfulness Evaluation for Diverse Summarisation Tasks Huajian Zhang Yumo Xu Laura Perez-Beltrachini HILM 34 10 0 27 Feb 2024
Successfully Guiding Humans with Imperfect Instructions by Highlighting Potential Errors and Suggesting Corrections Lingjun Zhao Khanh Nguyen Hal Daumé 37 1 0 26 Feb 2024
Entity-level Factual Adaptiveness of Fine-tuning based Abstractive Summarization Models Jongyoon Song Nohil Park Bongkyu Hwang Jaewoong Yun Seongho Joe Youngjune Gwon Sungroh Yoon KELM HILM 38 1 0 23 Feb 2024
Rethinking Scientific Summarization Evaluation: Grounding Explainable Metrics on Facet-aware Benchmark Preslav Nakov Tairan Wang Qingqing Zhu Taicheng Guo Shen Gao Zhiyong Lu Xin Gao Xiangliang Zhang 80 2 0 22 Feb 2024
SYNFAC-EDIT: Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarization Prakamya Mishra Zonghai Yao Parth Vashisht Feiyun Ouyang Beining Wang Vidhi Mody Hong-ye Yu SyDa MedIm 44 4 0 21 Feb 2024
Factual consistency evaluation of summarization in the Era of large language models Zheheng Luo Qianqian Xie Sophia Ananiadou HILM 35 1 0 21 Feb 2024
TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization Liyan Tang Igor Shalyminov Amy Wing-mei Wong Jon Burnsky Jake W. Vincent ... Hang Su Lijia Sun Yi Zhang Saab Mansour Kathleen McKeown HILM 29 45 0 20 Feb 2024
Identifying Factual Inconsistencies in Summaries: Grounding Model Inference via Task Taxonomy Liyan Xu Zhenlin Su Mo Yu Jin Xu Jinho D. Choi Jie Zhou Fei Liu HILM 43 2 0 20 Feb 2024
GenAudit: Fixing Factual Errors in Language Model Outputs with Evidence Kundan Krishna S. Ramprasad Prakhar Gupta Byron C. Wallace Zachary Chase Lipton Jeffrey P. Bigham HILM KELM SyDa 52 9 0 19 Feb 2024