Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?

9 May 2024

Papers citing "Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?"

31 / 31 papers shown

Title
SLearnLLM: A Self-Learning Framework for Efficient Domain-Specific Adaptation of Large Language Models Xiang Liu Zhaoxiang Liu Peng Wang Kohou Wang Huan Hu Kai Wang Shiguo Lian 115 0 0 23 May 2025
Personalizing Student-Agent Interactions Using Log-Contextualized Retrieval Augmented Generation (RAG) Clayton Cohn Surya Rayala Caitlin Snyder J. Fonteles Shruti Jain ... Ashwin T S Namrata Srivastava Menton Deweese Angela Eeds Gautam Biswas RALM 40 0 0 22 May 2025
KaFT: Knowledge-aware Fine-tuning for Boosting LLMs' Domain-specific Question-Answering Performance Qihuang Zhong Liang Ding Xiantao Cai Juhua Liu Bo Du Dacheng Tao 63 0 0 21 May 2025
The Hallucination Tax of Reinforcement Finetuning Linxin Song Taiwei Shi Jieyu Zhao HILM LRM 65 0 0 20 May 2025
BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMs Junxiao Yang Jinzhe Tu Haoran Liu Xiaoce Wang Chujie Zheng ... Caishun Chen Tiantian He Hongning Wang Yew-Soon Ong Minlie Huang LRM 67 0 0 18 May 2025
Critique-Guided Distillation: Improving Supervised Fine-tuning via Better Distillation Berkcan Kapusuzoglu Supriyo Chakraborty Chia-Hsuan Lee Sambit Sahu 62 0 0 16 May 2025
On the generalization of language models from in-context learning and finetuning: a controlled study Andrew Kyle Lampinen Arslan Chaudhry Stephanie Chan Cody Wild Diane Wan Alex Ku Jorg Bornschein Razvan Pascanu Murray Shanahan James L. McClelland 99 3 0 01 May 2025
UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models Boyang Xue Fei Mi Qi Zhu Hongru Wang Rui Wang Sheng Wang Erxin Yu Xuming Hu Kam-Fai Wong HILM 140 2 0 16 Dec 2024
Bridging the Visual Gap: Fine-Tuning Multimodal Models with Knowledge-Adapted Captions Moran Yanuka Assaf Ben-Kish Yonatan Bitton Idan Szpektor Raja Giryes VLM 72 2 0 13 Nov 2024
Improving Model Factuality with Fine-grained Critique-based Evaluator Yiqing Xie Wenxuan Zhou Pradyot Prakash Di Jin Yuning Mao ... Sinong Wang Han Fang Carolyn Rose Daniel Fried Hejia Zhang HILM 87 6 0 24 Oct 2024
ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains Yein Park Chanwoong Yoon Jungwoo Park Donghyeon Lee Minbyul Jeong Jaewoo Kang KELM 105 1 0 13 Oct 2024
Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study Over Open-ended Question Answering Yuan Sui Yufei He Zifeng Ding Bryan Hooi HILM RALM ELM 106 9 0 10 Oct 2024
MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization Yougang Lyu Lingyong Yan Zihan Wang Dawei Yin Pengjie Ren Maarten de Rijke Zhaochun Ren 91 8 0 10 Oct 2024
LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations Hadas Orgad Michael Toker Zorik Gekhman Roi Reichart Idan Szpektor Hadas Kotek Yonatan Belinkov HILM AIFin 92 37 0 03 Oct 2024
Prompt Obfuscation for Large Language Models David Pape Thorsten Eisenhofer Thorsten Eisenhofer Lea Schönherr AAML 87 4 0 17 Sep 2024
AI Sandbagging: Language Models can Strategically Underperform on Evaluations Teun van der Weij Felix Hofstätter Ollie Jaffe Samuel F. Brown Francis Rhys Ward ELM 69 27 0 11 Jun 2024
Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teaching Xiaoying Zhang Baolin Peng Ye Tian Jingyan Zhou Yipeng Zhang Haitao Mi Helen Meng CLL KELM 105 5 0 10 Jun 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 191 363 0 21 Mar 2024
Fine-tuning Language Models for Factuality Katherine Tian Eric Mitchell Huaxiu Yao Christopher D. Manning Chelsea Finn KELM HILM SyDa 57 172 0 14 Nov 2023
The False Promise of Imitating Proprietary LLMs Arnav Gudibande Eric Wallace Charles Burton Snell Xinyang Geng Hao Liu Pieter Abbeel Sergey Levine Dawn Song ALM 93 202 0 25 May 2023
Discovering Latent Knowledge in Language Models Without Supervision Collin Burns Haotian Ye Dan Klein Jacob Steinhardt 109 350 0 07 Dec 2022
Mass-Editing Memory in a Transformer Kevin Meng Arnab Sen Sharma A. Andonian Yonatan Belinkov David Bau KELM VLM 101 543 0 13 Oct 2022
A Review on Language Models as Knowledge Bases Badr AlKhamissi Millicent Li Asli Celikyilmaz Mona T. Diab Marjan Ghazvininejad KELM 73 185 0 12 Apr 2022
Locating and Editing Factual Associations in GPT Kevin Meng David Bau A. Andonian Yonatan Belinkov KELM 170 1,308 0 10 Feb 2022
Learning To Retrieve Prompts for In-Context Learning Ohad Rubin Jonathan Herzig Jonathan Berant VPVLM RALM 61 679 0 16 Dec 2021
SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization Philippe Laban Tobias Schnabel Paul N. Bennett Marti A. Hearst HILM 78 389 0 18 Nov 2021
Simple Entity-Centric Questions Challenge Dense Retrievers Christopher Sciavolino Zexuan Zhong Jinhyuk Lee Danqi Chen RALM 61 162 0 17 Sep 2021
$$Q^{2}$: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering$ $Q^{2}$ : Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering Or Honovich Leshem Choshen Roee Aharoni Ella Neeman Idan Szpektor Omri Abend HILM 57 139 0 16 Apr 2021
Evaluating the Factual Consistency of Abstractive Text Summarization Wojciech Kry'sciñski Bryan McCann Caiming Xiong R. Socher HILM 83 739 0 28 Oct 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 545 2,639 0 03 Sep 2019
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 182 8,067 0 16 Jun 2016