Continual Learning for Large Language Models: A Survey

2 February 2024

Papers citing "Continual Learning for Large Language Models: A Survey"

34 / 34 papers shown

Title
SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning Jinpeng Chen Runmin Cong Yuzhi Zhao Hongzheng Yang Guangneng Hu H. Ip Sam Kwong CLL KELM 83 0 0 05 May 2025
Multimodal Large Language Models for Medicine: A Comprehensive Survey Jiarui Ye Hao Tang LM&MA 89 0 0 29 Apr 2025
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining Jeffrey Li Mohammadreza Armandpour Iman Mirzadeh Sachin Mehta Vaishaal Shankar ... Samy Bengio Oncel Tuzel Mehrdad Farajtabar Hadi Pouransari Fartash Faghri CLL KELM 61 0 0 02 Apr 2025
LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation Zican Dong Junyi Li Jinhao Jiang Mingyu Xu Wayne Xin Zhao Bin Wang Xin Wu VLM 213 3 0 20 Feb 2025
Lightweight Online Adaption for Time Series Foundation Model Forecasts Thomas L. Lee William Toner Rajkarn Singh Artjom Joosem Martin Asenov AI4TS 38 0 0 18 Feb 2025
A Survey of Personalized Large Language Models: Progress and Future Directions Jiahong Liu Zexuan Qiu Zhongyang Li Quanyu Dai Jieming Zhu Minda Hu Menglin Yang Irwin King LM&MA 54 2 0 17 Feb 2025
Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning Jiapu Wang Kai Sun Linhao Luo Wei Wei Yongli Hu Alan Wee-Chung Liew Shirui Pan Baocai Yin 52 5 0 31 Dec 2024
Gradient Localization Improves Lifelong Pretraining of Language Models Jared Fernandez Yonatan Bisk Emma Strubell KELM 36 1 0 07 Nov 2024
DELIFT: Data Efficient Language model Instruction Fine Tuning Ishika Agarwal Krishnateja Killamsetty Lucian Popa Marina Danilevksy ALM VLM 56 2 0 07 Nov 2024
GitChameleon: Unmasking the Version-Switching Capabilities of Code Generation Models Nizar Islah Justine Gehring Diganta Misra Eilif B. Muller Irina Rish Terry Yue Zhuo Massimo Caccia SyDa 40 1 0 05 Nov 2024
Controlled Low-Rank Adaptation with Subspace Regularization for Continued Training on Large Language Models Yuheng Lu Bingshuo Qian Caixia Yuan Huixing Jiang Xiaojie Wang CLL 34 0 0 22 Oct 2024
Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation Junhong Wu Yang Zhao Yangyifan Xu Bing Liu Chengqing Zong CLL 40 1 0 17 Oct 2024
Recent Advances of Multimodal Continual Learning: A Comprehensive Survey Dianzhi Yu Xinni Zhang Yankai Chen Aiwei Liu Yifei Zhang Philip S. Yu Irwin King VLM CLL 44 9 0 07 Oct 2024
Towards LifeSpan Cognitive Systems Yu Wang Chi Han Tongtong Wu Xiaoxin He Wangchunshu Zhou ... Zexue He Wei Wang Gholamreza Haffari Heng Ji Julian McAuley KELM CLL 141 1 0 20 Sep 2024
Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement Le Yu Bowen Yu Haiyang Yu Fei Huang Yongbin Li MoMe 29 5 0 06 Aug 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 77 1 0 30 Jul 2024
UCIL: An Unsupervised Class Incremental Learning Approach for Sound Event Detection Yang Xiao Rohan Kumar Das 44 3 0 04 Jul 2024
Refiner: Restructure Retrieval Content Efficiently to Advance Question-Answering Capabilities Zhonghao Li Xuming Hu Aiwei Liu Kening Zheng S. Huang Hui Xiong RALM 115 8 0 17 Jun 2024
CLoG: Benchmarking Continual Learning of Image Generation Models Haotian Zhang Junting Zhou Haowei Lin Hang Ye Jianhua Zhu Zihao Wang Liangcai Gao Yizhou Wang Yitao Liang DiffM VLM 34 0 0 07 Jun 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou Shuaiwen Leon Song Jianlong Wu Liqiang Nie Guohao Li 45 6 0 07 Jun 2024
When LLMs Meet Cybersecurity: A Systematic Literature Review Jie Zhang Haoyu Bu Hui Wen Yu Chen Lun Li Hongsong Zhu 42 36 0 06 May 2024
HFT: Half Fine-Tuning for Large Language Models Tingfeng Hui Zhenyu Zhang Shuohuan Wang Weiran Xu Yu Sun Hua-Hong Wu CLL 37 4 0 29 Apr 2024
From Matching to Generation: A Survey on Generative Information Retrieval Xiaoxi Li Jiajie Jin Yujia Zhou Yuyao Zhang Peitian Zhang Yutao Zhu Zhicheng Dou 3DV 78 46 0 23 Apr 2024
A Survey on Self-Evolution of Large Language Models Zhengwei Tao Ting-En Lin Xiancai Chen Hangyu Li Yuchuan Wu Yongbin Li Zhi Jin Fei Huang Dacheng Tao Jingren Zhou LRM LM&Ro 54 22 0 22 Apr 2024
Double Mixture: Towards Continual Event Detection from Speech Jingqi Kang Tongtong Wu Jinming Zhao Guitao Wang Yinwei Wei Haomiao Yang Guilin Qi Yuan-Fang Li Gholamreza Haffari 28 0 0 20 Apr 2024
Hyperparameters in Continual Learning: A Reality Check Sungmin Cha Kyunghyun Cho CLL 71 2 0 14 Mar 2024
Investigating Continual Pretraining in Large Language Models: Insights and Implications cCaugatay Yildiz Nishaanth Kanna Ravichandran Prishruit Punia Matthias Bethge B. Ermiş CLL KELM LRM 55 25 0 27 Feb 2024
Efficient Continual Pre-training for Building Domain Specific Large Language Models Yong Xie Karan Aggarwal Aitzaz Ahmad CLL 29 21 0 14 Nov 2023
CITB: A Benchmark for Continual Instruction Tuning Zihan Zhang Meng Fang Ling-Hao Chen Mohammad-Reza Namazi-Rad ALM CLL 44 23 0 23 Oct 2023
Orthogonal Subspace Learning for Language Model Continual Learning Xiao Wang Tianze Chen Qiming Ge Han Xia Rong Bao Rui Zheng Qi Zhang Tao Gui Xuanjing Huang CLL 119 89 0 22 Oct 2023
Fine-tuned Language Models are Continual Learners Thomas Scialom Tuhin Chakrabarty Smaranda Muresan CLL LRM 145 117 0 24 May 2022
$Understanding Dataset Difficulty with $\mathcal{V}$-Usable Information$ Understanding Dataset Difficulty with $\mathcal{V}$ -Usable Information Kawin Ethayarajh Yejin Choi Swabha Swayamdipta 164 157 0 16 Oct 2021
Towards Continual Knowledge Learning of Language Models Joel Jang Seonghyeon Ye Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Stanley Jungkyu Choi Minjoon Seo CLL KELM 227 150 0 07 Oct 2021
Curriculum-Meta Learning for Order-Robust Continual Relation Extraction Tongtong Wu Xuekai Li Yuan-Fang Li Reza Haffari Guilin Qi Yujin Zhu Guoqiang Xu CLL 32 67 0 06 Jan 2021