A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation

8 July 2023

Wenlin Yao

Dong Yu

Papers citing "A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation"

50 / 120 papers shown

Title
Chaos with Keywords: Exposing Large Language Models Sycophantic Hallucination to Misleading Keywords and Evaluating Defense Strategies Aswin Rrv Nemika Tyagi Md Nayem Uddin Neeraj Varshney Chitta Baral 37 0 0 06 Jun 2024
Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning Xiaohu Du Ming Wen Jiahao Zhu Zifan Xie Bin Ji Huijun Liu Xuanhua Shi Hai Jin 37 14 0 06 Jun 2024
When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs Ryo Kamoi Yusen Zhang Nan Zhang Jiawei Han Rui Zhang LRM 50 57 0 03 Jun 2024
Luna: An Evaluation Foundation Model to Catch Language Model Hallucinations with High Accuracy and Low Cost Masha Belyi Robert Friel Shuai Shao Atindriyo Sanyal HILM RALM 64 5 0 03 Jun 2024
Evaluating Uncertainty-based Failure Detection for Closed-Loop LLM Planners Zhi Zheng Qian Feng Hang Li Alois C. Knoll Jianxiang Feng 54 6 0 01 Jun 2024
SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales Tianyang Xu Shujin Wu Shizhe Diao Xiaoze Liu Xingyao Wang Yangyi Chen Jing Gao LRM 29 27 0 31 May 2024
ANAH: Analytical Annotation of Hallucinations in Large Language Models Ziwei Ji Yuzhe Gu Wenwei Zhang Chengqi Lyu Dahua Lin Kai-xiang Chen HILM 56 2 0 30 May 2024
Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities Alexander Nikitin Jannik Kossen Yarin Gal Pekka Marttinen UQCV 53 23 0 30 May 2024
Evolutionary Large Language Model for Automated Feature Transformation Nanxu Gong Chandan K. Reddy Wangyang Ying Yanjie Fu 39 6 0 25 May 2024
Navigating LLM Ethics: Advancements, Challenges, and Future Directions Junfeng Jiao S. Afroogh Yiming Xu Connor Phillips AILaw 62 19 0 14 May 2024
Enhanced Language Model Truthfulness with Learnable Intervention and Uncertainty Expression Farima Fatahi Bayat Xin Liu H. V. Jagadish Lu Wang HILM KELM 33 2 0 01 May 2024
MetaCheckGPT -- A Multi-task Hallucination Detector Using LLM Uncertainty and Meta-models Rahul Mehta Andrew Hoblitzell Jack O’keefe Hyeju Jang Vasudeva Varma HILM KELM 19 0 0 10 Apr 2024
SLPL SHROOM at SemEval2024 Task 06: A comprehensive study on models ability to detect hallucination Pouya Fallah S. Gooran Mohammad Jafarinasab Pouya Sadeghi Reza Farnia Amirreza Tarabkhah Zainab Sadat Taghavi Hossein Sameti HILM 51 3 0 07 Apr 2024
Multicalibration for Confidence Scoring in LLMs Gianluca Detommaso Martín Bertrán Riccardo Fogliato Aaron Roth 29 12 0 06 Apr 2024
Hallucination Diversity-Aware Active Learning for Text Summarization Yu Xia Xu Liu Tong Yu Sungchul Kim Ryan A. Rossi Anup B. Rao Tung Mai Shuai Li HILM 40 3 0 02 Apr 2024
FACTOID: FACtual enTailment fOr hallucInation Detection Vipula Rawte S. M. Towhidul Krishnav Rajbangshi Shravani Nag Aman Chadha Amit P. Sheth Amitava Das HILM 42 3 0 28 Mar 2024
"Sorry, Come Again?" Prompting -- Enhancing Comprehension and Diminishing Hallucination with [PAUSE]-injected Optimal Paraphrasing Vipula Rawte Islam Tonmoy M. M. Zaman Prachi Priya Marcin Kardas Alan Schelten Ruan Silva LRM 28 1 0 27 Mar 2024
Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations Lei Yu Meng Cao Jackie Chi Kit Cheung Yue Dong HILM 33 7 0 27 Mar 2024
ChatGPT Incorrectness Detection in Software Reviews M. Tanzil Junaed Younus Khan Gias Uddin 19 4 0 25 Mar 2024
Hallucination Detection in Foundation Models for Decision-Making: A Flexible Definition and Review of the State of the Art Neeloy Chakraborty Melkior Ornik Katherine Driggs-Campbell LRM 57 9 0 25 Mar 2024
Truth-Aware Context Selection: Mitigating Hallucinations of Large Language Models Being Misled by Untruthful Contexts Tian Yu Shaolei Zhang Yang Feng HILM 34 7 0 12 Mar 2024
Unfamiliar Finetuning Examples Control How Language Models Hallucinate Katie Kang Eric Wallace Claire Tomlin Aviral Kumar Sergey Levine HILM LRM 41 49 0 08 Mar 2024
Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification Ekaterina Fadeeva Aleksandr Rubashevskii Artem Shelmanov Sergey Petrakov Haonan Li ... Gleb Kuzmin Alexander Panchenko Timothy Baldwin Preslav Nakov Maxim Panov HILM 43 38 0 07 Mar 2024
HaluEval-Wild: Evaluating Hallucinations of Language Models in the Wild Zhiying Zhu Yiming Yang Zhiqing Sun HILM VLM 43 14 0 07 Mar 2024
Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models Xinran Zhao Hongming Zhang Xiaoman Pan Wenlin Yao Dong Yu Tongshuang Wu Jianshu Chen HILM LRM 27 4 0 27 Feb 2024
Retrieve Only When It Needs: Adaptive Retrieval Augmentation for Hallucination Mitigation in Large Language Models Hanxing Ding Liang Pang Zihao Wei Huawei Shen Xueqi Cheng HILM RALM 81 15 0 16 Feb 2024
Comparing Hallucination Detection Metrics for Multilingual Generation Haoqiang Kang Terra Blevins Luke Zettlemoyer HILM 37 16 0 16 Feb 2024
Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation Xiaoying Zhang Baolin Peng Ye Tian Jingyan Zhou Lifeng Jin Linfeng Song Haitao Mi Helen Meng HILM 42 43 0 14 Feb 2024
EntGPT: Linking Generative Large Language Models with Knowledge Bases Yifan Ding Amrit Poudel Qingkai Zeng Tim Weninger Balaji Veeramani Sanmitra Bhattacharya ReLM KELM LRM 31 4 0 09 Feb 2024
Factuality of Large Language Models in the Year 2024 Yuxia Wang Minghan Wang Muhammad Arslan Manzoor Fei Liu Georgi Georgiev Rocktim Jyoti Das Preslav Nakov LRM HILM 35 7 0 04 Feb 2024
LLM-based NLG Evaluation: Current Status and Challenges Mingqi Gao Xinyu Hu Jie Ruan Xiao Pu Xiaojun Wan ELM LM&MA 60 29 0 02 Feb 2024
Redefining "Hallucination" in LLMs: Towards a psychology-informed framework for mitigating misinformation Elijah Berberette Jack Hutchins Amir Sadovnik 14 9 0 01 Feb 2024
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems Tianyu Cui Yanling Wang Chuanpu Fu Yong Xiao Sijia Li ... Junwu Xiong Xinyu Kong Zujie Wen Ke Xu Qi Li 57 56 0 11 Jan 2024
The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models Junyi Li Jie Chen Ruiyang Ren Xiaoxue Cheng Wayne Xin Zhao Jian-Yun Nie Ji-Rong Wen HILM 43 43 0 06 Jan 2024
A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models S.M. Towhidul Islam Tonmoy S. M. M. Zaman Vinija Jain Anku Rani Vipula Rawte Aman Chadha Amitava Das HILM 43 182 0 02 Jan 2024
RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models Cheng Niu Yuanhao Wu Juno Zhu Siliang Xu Kashun Shum Randy Zhong Juntong Song Tong Zhang HILM 28 86 0 31 Dec 2023
Context-aware Decoding Reduces Hallucination in Query-focused Summarization Zhichao Xu HILM 24 10 0 21 Dec 2023
The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation Rongwu Xu Brian S. Lin Shujian Yang Tianqi Zhang Weiyan Shi Lei Bai Zhixuan Fang Wei Xu Han Qiu 46 51 0 14 Dec 2023
HALO: An Ontology for Representing and Categorizing Hallucinations in Large Language Models Navapat Nananukul Mayank Kejriwal HILM 24 3 0 08 Dec 2023
A Survey of the Evolution of Language Model-Based Dialogue Systems Hongru Wang Lingzhi Wang Yiming Du Liang Chen Jing Zhou Yufei Wang Kam-Fai Wong LRM 59 20 0 28 Nov 2023
UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation Xun Liang Shichao Song Simin Niu Zhiyu Li Feiyu Xiong ... Zhaohui Wy Dawei He Peng Cheng Zhonghao Wang Haiying Deng HILM 34 19 0 26 Nov 2023
Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledge Genglin Liu Xingyao Wang Lifan Yuan Yangyi Chen Hao Peng 29 16 0 16 Nov 2023
R-Tuning: Instructing Large Language Models to Say `I Don't Know' Hanning Zhang Shizhe Diao Yong Lin Yi Ren Fung Qing Lian Xingyao Wang Yangyi Chen Heng Ji Tong Zhang UQLM 39 37 0 16 Nov 2023
Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectification Haoqiang Kang Juntong Ni Huaxiu Yao HILM LRM 32 33 0 15 Nov 2023
A Survey of Confidence Estimation and Calibration in Large Language Models Jiahui Geng Fengyu Cai Yuxia Wang Heinz Koeppl Preslav Nakov Iryna Gurevych UQCV 41 54 0 14 Nov 2023
Insights into Classifying and Mitigating LLMs' Hallucinations Alessandro Bruno P. Mazzeo Aladine Chetouani Marouane Tliba M. A. Kerkouri HILM 45 10 0 14 Nov 2023
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Lei Huang Weijiang Yu Weitao Ma Weihong Zhong Zhangyin Feng ... Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin Ting Liu LRM HILM 39 722 0 09 Nov 2023
SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency Jiaxin Zhang Zhuohang Li Kamalika Das Bradley Malin Kumar Sricharan HILM LRM 24 56 0 03 Nov 2023
Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity Cunxiang Wang Xiaoze Liu Yuanhao Yue Xiangru Tang Tianhang Zhang ... Linyi Yang Jindong Wang Xing Xie Zheng-Wei Zhang Yue Zhang HILM KELM 51 184 0 11 Oct 2023
The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediations Vipula Rawte Swagata Chakraborty Agnibh Pathak Anubhav Sarkar S.M. Towhidul Islam Tonmoy Aman Chadha Mikel Artetxe Punit Daniel Simig HILM 32 119 0 08 Oct 2023