From Parameters to Prompts: Understanding and Mitigating the Factuality Gap between Fine-Tuned LLMs

29 May 2025

Papers citing "From Parameters to Prompts: Understanding and Mitigating the Factuality Gap between Fine-Tuned LLMs"

27 / 27 papers shown

Title
TSDS: Data Selection for Task-Specific Model Finetuning Zifan Liu Amin Karbasi Theodoros Rekatsinas 54 5 0 15 Oct 2024
Understanding Finetuning for Factual Knowledge Extraction Gaurav R. Ghosal Tatsunori Hashimoto Aditi Raghunathan 56 15 0 20 Jun 2024
Knowledge Graph-Enhanced Large Language Models via Path Selection Haochen Liu Song Wang Yaochen Zhu Yushun Dong Jundong Li KELM 55 20 0 19 Jun 2024
Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? Zorik Gekhman G. Yona Roee Aharoni Matan Eyal Amir Feder Roi Reichart Jonathan Herzig 73 122 0 09 May 2024
Unfamiliar Finetuning Examples Control How Language Models Hallucinate Katie Kang Eric Wallace Claire Tomlin Aviral Kumar Sergey Levine HILM LRM 59 55 0 08 Mar 2024
LESS: Selecting Influential Data for Targeted Instruction Tuning Mengzhou Xia Sadhika Malladi Suchin Gururangan Sanjeev Arora Danqi Chen 109 210 0 06 Feb 2024
Supervised Knowledge Makes Large Language Models Better In-context Learners Linyi Yang Shuibai Zhang Zhuohao Yu Guangsheng Bao Yidong Wang ... Ruochen Xu Weirong Ye Xing Xie Weizhu Chen Yue Zhang 62 16 0 26 Dec 2023
Personas as a Way to Model Truthfulness in Language Models Nitish Joshi Javier Rando Abulhair Saparov Najoung Kim He He HILM 40 30 0 27 Oct 2023
Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review Banghao Chen Zhaofeng Zhang Nicolas Langrené Shengxin Zhu LLMAG 59 89 0 23 Oct 2023
Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learning Jiadong Wang Chengyu Wang Chuanqi Tan Jun Huang Ming Gao KELM 65 6 0 26 Sep 2023
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction Zeyuan Allen-Zhu Yuanzhi Li KELM 71 149 0 25 Sep 2023
Wikibio: a Semantic Resource for the Intersectional Analysis of Biographical Events M. Stranisci Rossana Damiano Enrico Mensa V. Patti Daniele P. Radicioni Tommaso Caselli 52 9 0 15 Jun 2023
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation Sewon Min Kalpesh Krishna Xinxi Lyu M. Lewis Wen-tau Yih Pang Wei Koh Mohit Iyyer Luke Zettlemoyer Hannaneh Hajishirzi HILM ALM 108 649 0 23 May 2023
Can We Edit Factual Knowledge by In-Context Learning? Ce Zheng Lei Li Qingxiu Dong Yuxuan Fan Zhiyong Wu Jingjing Xu Baobao Chang KELM 50 199 0 22 May 2023
Understanding Finetuning for Factual Knowledge Extraction from Language Models Mehran Kazemi Sid Mittal Deepak Ramachandran KELM 51 11 0 26 Jan 2023
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories Alex Troy Mallen Akari Asai Victor Zhong Rajarshi Das Daniel Khashabi Hannaneh Hajishirzi RALM HILM KELM 94 567 0 20 Dec 2022
Transformers learn in-context by gradient descent J. Oswald Eyvind Niklasson E. Randazzo João Sacramento A. Mordvintsev A. Zhmoginov Max Vladymyrov MLT 68 470 0 15 Dec 2022
What learning algorithm is in-context learning? Investigations with linear models Ekin Akyürek Dale Schuurmans Jacob Andreas Tengyu Ma Denny Zhou 54 468 0 28 Nov 2022
Large Language Models Struggle to Learn Long-Tail Knowledge Nikhil Kandpal H. Deng Adam Roberts Eric Wallace Colin Raffel RALM KELM 82 409 0 15 Nov 2022
Locating and Editing Factual Associations in GPT Kevin Meng David Bau A. Andonian Yonatan Belinkov KELM 154 1,308 0 10 Feb 2022
Simple Entity-Centric Questions Challenge Dense Retrievers Christopher Sciavolino Zexuan Zhong Jinhyuk Lee Danqi Chen RALM 45 162 0 17 Sep 2021
TruthfulQA: Measuring How Models Mimic Human Falsehoods Stephanie C. Lin Jacob Hilton Owain Evans HILM 87 1,825 0 08 Sep 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 64 3,678 0 03 Sep 2021
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika D. Song Jacob Steinhardt ELM RALM 135 4,222 0 07 Sep 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 474 41,106 0 28 May 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 528 2,639 0 03 Sep 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 633 11,979 0 27 Aug 2019