QuestEval: Summarization Asks for Fact-based Evaluation

23 March 2021

Papers citing "QuestEval: Summarization Asks for Fact-based Evaluation"

50 / 181 papers shown

Title
SEval-Ex: A Statement-Level Framework for Explainable Summarization Evaluation Tanguy Herserant Vincent Guigue ELM 40 0 0 04 May 2025
Evaluating and Mitigating Bias in AI-Based Medical Text Generation Xiuying Chen Tairan Wang Juexiao Zhou Zirui Song Xin Gao Xuzhi Zhang MedIm 44 1 0 24 Apr 2025
From Speech to Summary: A Comprehensive Survey of Speech Summarization Fabian Retkowski Maike Züfle Andreas Sudmann Dinah Pfau Jan Niehues Alexander Waibel 46 0 0 10 Apr 2025
AILS-NTUA at SemEval-2025 Task 3: Leveraging Large Language Models and Translation Strategies for Multilingual Hallucination Detection Dimitra Karkani Maria Lymperaiou Giorgos Filandrianos Nikolaos Spanos Athanasios Voulodimos Giorgos Stamou HILM LRM 86 0 0 04 Mar 2025
A Meta-Evaluation of Style and Attribute Transfer Metrics Amalie Brogaard Pauli Isabelle Augenstein Ira Assent 71 0 0 20 Feb 2025
SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation Song Duong Florian Le Bronnec Alexandre Allauzen Vincent Guigue Alberto Lumbreras Laure Soulier Patrick Gallinari HILM 50 0 0 20 Feb 2025
Reference-free Evaluation Metrics for Text Generation: A Survey Takumi Ito Kees van Deemter Jun Suzuki ELM 41 2 0 21 Jan 2025
SteLLA: A Structured Grading System Using LLMs with RAG Hefei Qiu Brian White Ashley Ding Reinaldo Costa Ali Hachem Wei Ding Ping Chen AI4Ed 61 0 0 17 Jan 2025
A review of faithfulness metrics for hallucination assessment in Large Language Models Ben Malin Tatiana Kalganova Nikoloas Boulgouris HILM 59 2 0 03 Jan 2025
Fine-grained and Explainable Factuality Evaluation for Multimodal Summarization Liqiang Jing Jingxuan Zuo Yue Zhang 47 7 0 31 Dec 2024
QAPyramid: Fine-grained Evaluation of Content Selection for Text Summarization Shiyue Zhang David Wan Arie Cattan Ayal Klein Ido Dagan Joey Tianyi Zhou 86 0 0 10 Dec 2024
I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token Roi Cohen Konstantin Dobler Eden Biran Gerard de Melo 93 3 0 09 Dec 2024
Is my Meeting Summary Good? Estimating Quality with a Multi-LLM Evaluator Frederic Kirstein Terry Ruas Bela Gipp 87 2 0 27 Nov 2024
Do Automatic Factuality Metrics Measure Factuality? A Critical Evaluation S. Ramprasad Byron C. Wallace LLMAG HILM 87 2 0 25 Nov 2024
From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization Catarina G. Belem Pouya Pezeskhpour Hayate Iso Seiji Maekawa Nikita Bhutani Estevam R. Hruschka HILM 73 1 0 17 Oct 2024
LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations Hadas Orgad Michael Toker Zorik Gekhman Roi Reichart Idan Szpektor Hadas Kotek Yonatan Belinkov HILM AIFin 61 29 0 03 Oct 2024
Probing Omissions and Distortions in Transformer-based RDF-to-Text Models J. Faille Albert Gatt Claire Gardent 34 0 0 25 Sep 2024
Using Similarity to Evaluate Factual Consistency in Summaries Yuxuan Ye Edwin Simpson Raul Santos Rodriguez HILM 23 2 0 23 Sep 2024
Improving Clinical Note Generation from Complex Doctor-Patient Conversation Yizhan Li Sifan Wu Christopher W. Smith Thomas Lo Bang Liu 36 3 0 26 Aug 2024
A Comparative Analysis of Faithfulness Metrics and Humans in Citation Evaluation Weijia Zhang Mohammad Aliannejadi Jiahuan Pei Yifei Yuan Jia-Hong Huang Evangelos Kanoulas HILM 45 4 0 22 Aug 2024
uMedSum: A Unified Framework for Advancing Medical Abstractive Summarization Aishik Nagar Yutong Liu Andy T. Liu Viktor Schlegel Vijay Prakash Dwivedi Arun-Kumar Kaliya-Perumal Guna Pratheep Kalanchiam Yili Tang R. Tan 29 1 0 22 Aug 2024
Zero-shot Factual Consistency Evaluation Across Domains Raunak Agarwal HILM 47 0 0 07 Aug 2024
Towards Dataset-scale and Feature-oriented Evaluation of Text Summarization in Large Language Model Prompts Sam Yu-Te Lee Aryaman Bahukhandi Dongyu Liu Kwan-Liu Ma AAML 42 5 0 16 Jul 2024
LLM-Collaboration on Automatic Science Journalism for the General Audience Gongyao Jiang Xinran Shi Qiong Luo 29 3 0 13 Jul 2024
Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models Yuyan Chen Qiang Fu Yichen Yuan Zhihao Wen Ge Fan Dayiheng Liu Dongmei Zhang Zhixu Li Yanghua Xiao HILM 49 69 0 04 Jul 2024
LLM Internal States Reveal Hallucination Risk Faced With a Query Ziwei Ji Delong Chen Etsuko Ishii Samuel Cahyawijaya Yejin Bang Bryan Wilie Pascale Fung HILM LRM 39 20 0 03 Jul 2024
FineSurE: Fine-grained Summarization Evaluation using LLMs Hwanjun Song Hang Su Igor Shalyminov Jason (Jinglun) Cai Saab Mansour HILM 41 31 0 01 Jul 2024
A Comparative Study of Quality Evaluation Methods for Text Summarization Huyen Nguyen Haihua Chen Lavanya Pobbathi Junhua Ding ELM 40 5 0 30 Jun 2024
Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics Weijia Zhang Mohammad Aliannejadi Yifei Yuan Jiahuan Pei Jia-Hong Huang Evangelos Kanoulas HILM 31 12 0 21 Jun 2024
Factual Dialogue Summarization via Learning from Large Language Models Rongxin Zhu Jey Han Lau Jianzhong Qi HILM 52 1 0 20 Jun 2024
A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models Haopeng Zhang Philip S. Yu Jiawei Zhang 37 17 0 17 Jun 2024
Key-Element-Informed sLLM Tuning for Document Summarization Sangwon Ryu Heejin Do Yunsu Kim G. G. Lee Jungseul Ok 29 6 0 07 Jun 2024
Analyzing LLM Behavior in Dialogue Summarization: Unveiling Circumstantial Hallucination Trends S. Ramprasad Elisa Ferracane Zachary Chase Lipton HILM 19 12 0 05 Jun 2024
When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs Ryo Kamoi Yusen Zhang Nan Zhang Jiawei Han Rui Zhang LRM 50 57 0 03 Jun 2024
Multi-Dimensional Optimization for Text Summarization via Reinforcement Learning Sangwon Ryu Heejin Do Yunsu Kim Gary Geunbae Lee Jungseul Ok 28 3 0 01 Jun 2024
Faithful Chart Summarization with ChaTS-Pi Syrine Krichene Francesco Piccinno Fangyu Liu Julian Martin Eisenschlos 37 1 0 29 May 2024
Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? Zorik Gekhman G. Yona Roee Aharoni Matan Eyal Amir Feder Roi Reichart Jonathan Herzig 52 104 0 09 May 2024
Natural Language Processing RELIES on Linguistics Juri Opitz Shira Wein Nathan Schneider AI4CE 55 7 0 09 May 2024
Large Language Models are Inconsistent and Biased Evaluators Rickard Stureborg Dimitris Alikaniotis Yoshi Suhara ALM 47 51 0 02 May 2024
ISQA: Informative Factuality Feedback for Scientific Summarization Zekai Li Yanxia Qin Qian Liu Min-Yen Kan HILM 37 1 0 20 Apr 2024
Can We Catch the Elephant? A Survey of the Evolvement of Hallucination Evaluation on Natural Language Generation Siya Qi Yulan He Zheng Yuan LRM HILM 43 1 0 18 Apr 2024
FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document Joonho Yang Seunghyun Yoon Byeongjeong Kim Hwanhee Lee HILM 34 4 0 17 Apr 2024
Less is More for Improving Automatic Evaluation of Factual Consistency Tong Wang Ninad Kulkarni Yanjun Qi ALM 49 2 0 09 Apr 2024
The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Models Giwon Hong Aryo Pradipta Gema Rohit Saxena Xiaotang Du Ping Nie ... Laura Perez-Beltrachini Max Ryabinin Xuanli He Clémentine Fourrier Pasquale Minervini LRM HILM 38 11 0 08 Apr 2024
Fakes of Varying Shades: How Warning Affects Human Perception and Engagement Regarding LLM Hallucinations Mahjabin Nahar Haeseung Seo Eun-Ju Lee Aiping Xiong Dongwon Lee HILM 37 11 0 04 Apr 2024
Evaluating Document Simplification: On the Importance of Separately Assessing Simplicity and Meaning Preservation Liam Cripwell Joël Legrand Claire Gardent 28 3 0 04 Apr 2024
CoUDA: Coherence Evaluation via Unified Data Augmentation Dawei Zhu Wenhao Wu Yifan Song Fangwei Zhu Ziqiang Cao Sujian Li 28 0 0 31 Mar 2024
REFeREE: A REference-FREE Model-Based Metric for Text Simplification Yichen Huang Ekaterina Kochmar 58 1 0 26 Mar 2024
A Closer Look at Claim Decomposition Miriam Wanner Seth Ebner Zhengping Jiang Mark Dredze Benjamin Van Durme 49 18 0 18 Mar 2024
SIFiD: Reassess Summary Factual Inconsistency Detection with LLM Jiuding Yang Hui Liu Weidong Guo Zhuwei Rao Yu-Syuan Xu Di Niu HILM 21 0 0 12 Mar 2024