An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning

An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning

17 August 2023

Papers citing "An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning"

16 / 66 papers shown

Title
JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capability Junda Wang Zhichao Yang Zonghai Yao Hong-ye Yu BDL AI4MH LRM 40 30 0 27 Feb 2024
Investigating Continual Pretraining in Large Language Models: Insights and Implications cCaugatay Yildiz Nishaanth Kanna Ravichandran Prishruit Punia Matthias Bethge Beyza Ermis CLL KELM LRM 58 25 0 27 Feb 2024
Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning Zhaorui Yang Tianyu Pang H. Feng Han Wang Wei Chen Minfeng Zhu Qian Liu ALM 37 35 0 21 Feb 2024
RefuteBench: Evaluating Refuting Instruction-Following for Large Language Models Jianhao Yan Yun Luo Yue Zhang ALM LRM 38 6 0 21 Feb 2024
CoLLaVO: Crayon Large Language and Vision mOdel Byung-Kwan Lee Beomchan Park Chae Won Kim Yonghyun Ro VLM MLLM 32 16 0 17 Feb 2024
Improving Sequential Recommendations with LLMs Artun Boz Wouter Zorgdrager Zoe Kotti Jesse Harte Panos Louridas Dietmar Jannach Vassilios Karakoidas Marios Fragkoulis KELM LRM 70 4 0 02 Feb 2024
LDM-ISP: Enhancing Neural ISP for Low Light with Latent Diffusion Models Qiang Wen Yazhou Xing Zhefan Rao Qifeng Chen DiffM 35 0 0 02 Dec 2023
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 25 27 0 15 Nov 2023
Instruction Tuning with Human Curriculum Bruce W. Lee Hyunsoo Cho Kang Min Yoo 45 3 0 14 Oct 2023
Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model Qichen Ye Junling Liu Dading Chong Peilin Zhou Yining Hua ... Meng Cao Ziming Wang Xuxin Cheng Andrew Liu Zhenhua Guo AI4MH LM&MA ELM 30 20 0 13 Oct 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 301 2,232 0 22 Mar 2023
Continual Training of Language Models for Few-Shot Learning Zixuan Ke Haowei Lin Yijia Shao Hu Xu Lei Shu Bin Liu KELM BDL CLL 90 34 0 11 Oct 2022
Fine-tuned Language Models are Continual Learners Thomas Scialom Tuhin Chakrabarty Smaranda Muresan CLL LRM 145 117 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 330 11,953 0 04 Mar 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021
e-SNLI: Natural Language Inference with Natural Language Explanations Oana-Maria Camburu Tim Rocktaschel Thomas Lukasiewicz Phil Blunsom LRM 260 620 0 04 Dec 2018