How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis

31 March 2022

Lifeng Shang

Chengjie Sun

Xin Jiang

Qun Liu

KELM

ArXiv PDF HTML

Papers citing "How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis"

35 / 35 papers shown

Title
Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations Yiyou Sun Y. Gai Lijie Chen Abhilasha Ravichander Yejin Choi D. Song HILM 57 0 0 17 Apr 2025
KSHSeek: Data-Driven Approaches to Mitigating and Detecting Knowledge-Shortcut Hallucinations in Generative Models Z. Wang Zhongxin Liu Ying Li Hongyu Sun Meng Xu Yuqing Zhang HILM 53 0 0 25 Mar 2025
BiasEdit: Debiasing Stereotyped Language Models via Model Editing Xin Xu Wei Xu N. Zhang Julian McAuley KELM 39 0 0 11 Mar 2025
Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge Xinyue Cui Johnny Tian-Zheng Wei Swabha Swayamdipta Robin Jia WaLM 91 1 0 06 Mar 2025
IPL: Leveraging Multimodal Large Language Models for Intelligent Product Listing Kang Chen Qingheng Zhang Chengbao Lian Yixin Ji Xuwei Liu Shuguang Han Guoqiang Wu Fei Huang Jufeng Chen 31 1 0 22 Oct 2024
On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs Nitay Calderon Roi Reichart 40 10 0 27 Jul 2024
Knowledge Mechanisms in Large Language Models: A Survey and Perspective Meng Wang Yunzhi Yao Ziwen Xu Shuofei Qiao Shumin Deng ... Yong-jia Jiang Pengjun Xie Fei Huang Huajun Chen Ningyu Zhang 55 28 0 22 Jul 2024
Look Within, Why LLMs Hallucinate: A Causal Perspective He Li Haoang Chi Mingyu Liu Wenjing Yang LRM 37 4 0 14 Jul 2024
How Do Large Language Models Acquire Factual Knowledge During Pretraining? Hoyeon Chang Jinho Park Seonghyeon Ye Sohee Yang Youngkyung Seo Du-Seong Chang Minjoon Seo KELM 37 32 0 17 Jun 2024
ProbGate at EHRSQL 2024: Enhancing SQL Query Generation Accuracy through Probabilistic Threshold Filtering and Error Handling Sangryul Kim Donghee Han Sehyun Kim 21 3 0 25 Apr 2024
Knowledge Conflicts for LLMs: A Survey Rongwu Xu Zehan Qi Zhijiang Guo Cunxiang Wang Hongru Wang Yue Zhang Wei Xu 200 94 0 13 Mar 2024
InfuserKI: Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integration Fali Wang Runxue Bao Suhang Wang Wenchao Yu Yanchi Liu Wei Cheng Haifeng Chen KELM 21 8 0 18 Feb 2024
The Queen of England is not England's Queen: On the Lack of Factual Coherency in PLMs Paul Youssef Jorg Schlotterer Christin Seifert KELM 26 2 0 02 Feb 2024
BayesPrompt: Prompting Large-Scale Pre-Trained Language Models on Few-shot Inference via Debiased Domain Abstraction Jiangmeng Li Fei Song Yifan Jin Jingyao Wang Changwen Zheng Gang Hua Hui Xiong VLM 42 2 0 25 Jan 2024
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems Tianyu Cui Yanling Wang Chuanpu Fu Yong Xiao Sijia Li ... Junwu Xiong Xinyu Kong Zujie Wen Ke Xu Qi Li 57 56 0 11 Jan 2024
The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models Junyi Li Jie Chen Ruiyang Ren Xiaoxue Cheng Wayne Xin Zhao Jian-Yun Nie Ji-Rong Wen HILM 43 43 0 06 Jan 2024
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Lei Huang Weijiang Yu Weitao Ma Weihong Zhong Zhangyin Feng ... Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin Ting Liu LRM HILM 39 722 0 09 Nov 2023
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models Paul Youssef Osman Alperen Koracs Meijie Li Jorg Schlotterer Christin Seifert KELM 24 16 0 25 Oct 2023
Impact of Co-occurrence on Factual Knowledge of Large Language Models Cheongwoong Kang Jaesik Choi KELM 26 17 0 12 Oct 2023
Faithful Knowledge Graph Explanations for Commonsense Reasoning Weihe Zhai A. Zubiaga 22 0 0 07 Oct 2023
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models Yue Zhang Yafu Li Leyang Cui Deng Cai Lemao Liu ... Longyue Wang A. Luu Wei Bi Freda Shi Shuming Shi RALM LRM HILM 46 520 0 03 Sep 2023
Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons Yuheng Chen Pengfei Cao Yubo Chen Kang Liu Jun Zhao KELM 25 41 0 25 Aug 2023
On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook Mingyuan Fan Chengyu Wang Cen Chen Yang Liu Jun Huang HILM 36 3 0 31 Jul 2023
Generating Benchmarks for Factuality Evaluation of Language Models Dor Muhlgay Ori Ram Inbal Magar Yoav Levine Nir Ratner Yonatan Belinkov Omri Abend Kevin Leyton-Brown Amnon Shashua Y. Shoham HILM 25 91 0 13 Jul 2023
Unifying Large Language Models and Knowledge Graphs: A Roadmap Shirui Pan Linhao Luo Yufei Wang Chen Chen Jiapu Wang Xindong Wu KELM 35 715 0 14 Jun 2023
Examining the Emergence of Deductive Reasoning in Generative Language Models Peter Belcak Luca A. Lanzendörfer Roger Wattenhofer ReLM ELM LRM AI4CE 13 0 0 31 May 2023
Images in Language Space: Exploring the Suitability of Large Language Models for Vision & Language Tasks Sherzod Hakimov David Schlangen VLM 36 5 0 23 May 2023
"According to ...": Prompting Language Models Improves Quoting from Pre-Training Data Orion Weller Marc Marone Nathaniel Weir Dawn J Lawrie Daniel Khashabi Benjamin Van Durme HILM 78 44 0 22 May 2023
Causality-aware Concept Extraction based on Knowledge-guided Prompting Siyu Yuan Deqing Yang Jinxi Liu Shuyu Tian Jiaqing Liang Yanghua Xiao R. Xie 66 13 0 03 May 2023
The Life Cycle of Knowledge in Big Language Models: A Survey Boxi Cao Hongyu Lin Xianpei Han Le Sun KELM 33 27 0 14 Mar 2023
A Review of the Role of Causality in Developing Trustworthy AI Systems Niloy Ganguly Dren Fazlija Maryam Badar M. Fisichella Sandipan Sikdar ... Koustav Rudra Manolis Koubarakis Gourab K. Patro W. Z. E. Amri Wolfgang Nejdl CML 39 23 0 14 Feb 2023
A Survey of Knowledge Enhanced Pre-trained Language Models Linmei Hu Zeyi Liu Ziwang Zhao Lei Hou Liqiang Nie Juanzi Li KELM VLM 24 121 0 11 Nov 2022
Towards Faithful Model Explanation in NLP: A Survey Qing Lyu Marianna Apidianaki Chris Callison-Burch XAI 109 107 0 22 Sep 2022
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 269 346 0 01 Feb 2021
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 417 2,588 0 03 Sep 2019