Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations

17 April 2025

Yiyou Sun

Y. Gai

Lijie Chen

Abhilasha Ravichander

Papers citing "Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations"

33 / 33 papers shown

Title
Invisible Tokens, Visible Bills: The Urgent Need to Audit Hidden Operations in Opaque LLM Services Guoheng Sun Ziyao Wang Xuandong Zhao Bowei Tian Zheyu Shen Yexiao He Jinming Xing Ang Li 63 0 0 24 May 2025
HALoGEN: Fantastic LLM Hallucinations and Where to Find Them Abhilasha Ravichander Shrusti Ghela David Wadden Yejin Choi HILM LRM 106 5 0 14 Jan 2025
Understanding Factual Recall in Transformers via Associative Memories Eshaan Nichani Jason D. Lee Alberto Bietti KELM 96 12 0 09 Dec 2024
The BrowserGym Ecosystem for Web Agent Research Thibault Le Sellier De Chezelles Maxime Gasse Alexandre Lacoste Alexandre Drouin Massimo Caccia ... Siva Reddy Quentin Cappart Graham Neubig Ruslan Salakhutdinov Nicolas Chapados LLMAG 160 18 0 06 Dec 2024
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research Luca Soldaini Rodney Michael Kinney Akshita Bhagia Dustin Schwenk David Atkinson ... Hanna Hajishirzi Iz Beltagy Dirk Groeneveld Jesse Dodge Kyle Lo 92 278 0 31 Jan 2024
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models Yue Zhang Yafu Li Leyang Cui Deng Cai Lemao Liu ... Longyue Wang Anh Tuan Luu Wei Bi Freda Shi Shuming Shi RALM LRM HILM 106 577 0 03 Sep 2023
Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs? Kai Sun Yongjun Xu Hanwen Zha Yue Liu Xinhsuai Dong AI4MH 100 148 0 20 Aug 2023
Simple synthetic data reduces sycophancy in large language models Jerry W. Wei Da Huang Yifeng Lu Denny Zhou Quoc V. Le 89 73 0 07 Aug 2023
WebArena: A Realistic Web Environment for Building Autonomous Agents Shuyan Zhou Frank F. Xu Hao Zhu Xuhui Zhou Robert Lo ... Tianyue Ou Yonatan Bisk Daniel Fried Uri Alon Graham Neubig LLMAG 176 490 0 25 Jul 2023
Do Large Language Models Know What They Don't Know? Zhangyue Yin Qiushi Sun Qipeng Guo Jiawen Wu Xipeng Qiu Xuanjing Huang ELM AI4MH 78 163 0 29 May 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.5K 14,699 0 15 Mar 2023
Impossibility Theorems for Feature Attribution Blair Bilodeau Natasha Jaques Pang Wei Koh Been Kim FAtt 65 77 0 22 Dec 2022
Transformers learn in-context by gradient descent J. Oswald Eyvind Niklasson E. Randazzo João Sacramento A. Mordvintsev A. Zhmoginov Max Vladymyrov MLT 116 494 0 15 Dec 2022
On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models? Nouha Dziri Sivan Milton Mo Yu Osmar Zaiane Siva Reddy HILM 49 194 0 17 Apr 2022
A Comparative Study of Faithfulness Metrics for Model Interpretability Methods Chun Sik Chan Huanqi Kong Guanqing Liang 79 53 0 12 Apr 2022
How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis Shaobo Li Xiaoguang Li Lifeng Shang Zhenhua Dong Chengjie Sun Bingquan Liu Zhenzhou Ji Xin Jiang Qun Liu KELM 86 55 0 31 Mar 2022
The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective Satyapriya Krishna Tessa Han Alex Gu Steven Wu S. Jabbari Himabindu Lakkaraju 246 195 0 03 Feb 2022
Logic Traps in Evaluating Attribution Scores Yiming Ju Yuanzhe Zhang Zhao Yang Zhongtao Jiang Kang Liu Jun Zhao XAI FAtt 79 19 0 12 Sep 2021
Discretized Integrated Gradients for Explaining Language Models Soumya Sanyal Xiang Ren FAtt 50 54 0 31 Aug 2021
Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models Mengnan Du Varun Manjunatha R. Jain Ruchi Deshpande Franck Dernoncourt Jiuxiang Gu Tong Sun Xia Hu 99 107 0 11 Mar 2021
Explaining by Removing: A Unified Framework for Model Explanation Ian Covert Scott M. Lundberg Su-In Lee FAtt 109 252 0 21 Nov 2020
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models Lifu Tu Garima Lalwani Spandana Gella He He LRM 96 187 0 14 Jul 2020
Beyond Accuracy: Behavioral Testing of NLP models with CheckList Marco Tulio Ribeiro Tongshuang Wu Carlos Guestrin Sameer Singh ELM 210 1,107 0 08 May 2020
Learning to Faithfully Rationalize by Construction Sarthak Jain Sarah Wiegreffe Yuval Pinter Byron C. Wallace 82 164 0 30 Apr 2020
Shortcut Learning in Deep Neural Networks Robert Geirhos J. Jacobsen Claudio Michaelis R. Zemel Wieland Brendel Matthias Bethge Felix Wichmann 214 2,059 0 16 Apr 2020
When Explanations Lie: Why Many Modified BP Attributions Fail Leon Sixt Maximilian Granz Tim Landgraf BDL FAtt XAI 58 132 0 20 Dec 2019
Neural Network Attributions: A Causal Perspective Aditya Chattopadhyay Piyushi Manupriya Anirban Sarkar V. Balasubramanian CML 65 146 0 06 Feb 2019
Sanity Checks for Saliency Maps Julius Adebayo Justin Gilmer M. Muelly Ian Goodfellow Moritz Hardt Been Kim FAtt AAML XAI 148 1,969 0 08 Oct 2018
A Unified Approach to Interpreting Model Predictions Scott M. Lundberg Su-In Lee FAtt 1.1K 22,018 0 22 May 2017
Axiomatic Attribution for Deep Networks Mukund Sundararajan Ankur Taly Qiqi Yan OOD FAtt 193 6,018 0 04 Mar 2017
"Why Should I Trust You?": Explaining the Predictions of Any Classifier Marco Tulio Ribeiro Sameer Singh Carlos Guestrin FAtt FaML 1.2K 17,033 0 16 Feb 2016
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks Samy Bengio Oriol Vinyals Navdeep Jaitly Noam M. Shazeer 152 2,038 0 09 Jun 2015
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 578 27,327 0 01 Sep 2014