Continual Learning for Large Language Models: A Survey

2 February 2024

Papers citing "Continual Learning for Large Language Models: A Survey"

50 / 83 papers shown

Title
SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning Jinpeng Chen Runmin Cong Yuzhi Zhao Hongzheng Yang Guangneng Hu H. Ip Sam Kwong CLL KELM 83 0 0 05 May 2025
Aligning Large Language Models with Healthcare Stakeholders: A Pathway to Trustworthy AI Integration Kexin Ding Mu Zhou Akshay Chaudhari Shaoting Zhang Dimitris N. Metaxas LM&MA 43 0 0 02 May 2025
Multimodal Large Language Models for Medicine: A Comprehensive Survey Jiarui Ye Hao Tang LM&MA 91 0 0 29 Apr 2025
Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Survey Ahsan Bilal Muhammad Ahmed Mohsin Muhammad Umer Muhammad Awais Khan Bangash Muhammad Ali Jamshed LLMAG LRM AI4CE 56 0 0 20 Apr 2025
SEE: Continual Fine-tuning with Sequential Ensemble of Experts Zhilin Wang Yafu Li Xiaoye Qu Yu Cheng CLL KELM 58 0 0 09 Apr 2025
Memory-Statistics Tradeoff in Continual Learning with Structural Regularization Haoran Li Jingfeng Wu Vladimir Braverman CLL 34 0 0 05 Apr 2025
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining Jeffrey Li Mohammadreza Armandpour Iman Mirzadeh Sachin Mehta Vaishaal Shankar ... Samy Bengio Oncel Tuzel Mehrdad Farajtabar Hadi Pouransari Fartash Faghri CLL KELM 61 0 0 02 Apr 2025
LLaVA-CMoE: Towards Continual Mixture of Experts for Large Vision-Language Models Hengyuan Zhao Ziqin Wang Qixin Sun Kaiyou Song Yilin Li Xiaolin Hu Qingpei Guo Si Liu KELM CLL MoE 65 0 0 27 Mar 2025
Enhancing Domain-Specific Encoder Models with LLM-Generated Data: How to Leverage Ontologies, and How to Do Without Them Marc Felix Brinner Tarek Al Mustafa Sina Zarrieß 39 0 0 27 Mar 2025
AfroXLMR-Social: Adapting Pre-trained Language Models for African Languages Social Media Text Tadesse Destaw Belay Israel Abebe Azime I. Ahmad Idris Abdulmumin A. Ayele Shamsuddeen Hassan Muhammad Seid Muhie Yimam 33 0 0 24 Mar 2025
Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities Weixiang Zhao Xingyu Sui Jiahe Guo Yulin Hu Yang Deng Yanyan Zhao Bing Qin Wanxiang Che Tat-Seng Chua Ting Liu ELM LRM 59 4 0 23 Mar 2025
Analytic Subspace Routing: How Recursive Least Squares Works in Continual Learning of Large Language Model Kai Tong Kang Pan Xiao Zhang Erli Meng Run He Yawen Cui Nuoyan Guo Huiping Zhuang KELM CLL 62 0 0 17 Mar 2025
DETQUS: Decomposition-Enhanced Transformers for QUery-focused Summarization Yasir Khan Xinlei Wu Sangpil Youm Justin Ho Aryaan Shaikh Jairo Garciga Rohan Sharma Bonnie J. Dorr LMTD 85 0 0 07 Mar 2025
LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation Zican Dong Junyi Li Jinhao Jiang Mingyu Xu Wayne Xin Zhao Bin Wang Xin Wu VLM 213 4 0 20 Feb 2025
Lightweight Online Adaption for Time Series Foundation Model Forecasts Thomas L. Lee William Toner Rajkarn Singh Artjom Joosem Martin Asenov AI4TS 38 0 0 18 Feb 2025
Mitigating Visual Knowledge Forgetting in MLLM Instruction-tuning via Modality-decoupled Gradient Descent Junda Wu Yuxin Xiong Xintong Li Yu Xia Ruoyu Wang ... Sungchul Kim Ryan Rossi Lina Yao Jingbo Shang Julian McAuley CLL VLM 57 0 0 17 Feb 2025
A Survey of Personalized Large Language Models: Progress and Future Directions Jiahong Liu Zexuan Qiu Zhongyang Li Quanyu Dai Jieming Zhu Minda Hu Menglin Yang Irwin King LM&MA 54 3 0 17 Feb 2025
Primus: A Pioneering Collection of Open-Source Datasets for Cybersecurity LLM Training Yao-Ching Yu Tsun-Han Chiang Cheng-Wei Tsai Chien-Ming Huang Wen-Kwang Tsao 62 6 0 16 Feb 2025
Continually Evolved Multimodal Foundation Models for Cancer Prognosis Jie Peng Shuang Zhou Longwei Yang Yiran Song Mohan Zhang Kaixiong Zhou Feng Xie Mingquan Lin Rui Zhang Tianlong Chen 90 0 0 30 Jan 2025
Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning Jiapu Wang Kai Sun Linhao Luo Wei Wei Yongli Hu Alan Wee-Chung Liew Shirui Pan Baocai Yin 52 6 0 31 Dec 2024
A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine Hanguang Xiao Feizhong Zhou X. Liu Tianqi Liu Zhipeng Li Xin Liu Xiaoxuan Huang AILaw LM&MA LRM 61 18 0 31 Dec 2024
Continual Memorization of Factoids in Language Models Howard Chen Jiayi Geng Adithya Bhaskar Dan Friedman Danqi Chen KELM 56 0 0 11 Nov 2024
Gradient Localization Improves Lifelong Pretraining of Language Models Jared Fernandez Yonatan Bisk Emma Strubell KELM 39 1 0 07 Nov 2024
DELIFT: Data Efficient Language model Instruction Fine Tuning Ishika Agarwal Krishnateja Killamsetty Lucian Popa Marina Danilevksy ALM VLM 58 3 0 07 Nov 2024
GitChameleon: Unmasking the Version-Switching Capabilities of Code Generation Models Nizar Islah Justine Gehring Diganta Misra Eilif B. Muller Irina Rish Terry Yue Zhuo Massimo Caccia SyDa 45 1 0 05 Nov 2024
Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models Haritz Puerto Martin Gubri Sangdoo Yun Seong Joon Oh MIALM 609 2 2 31 Oct 2024
TransformLLM: Adapting Large Language Models via LLM-Transformed Reading Comprehension Text Iftach Arbel Yehonathan Refael Ofir Lindenbaum AILaw 26 0 0 28 Oct 2024
Self-Normalized Resets for Plasticity in Continual Learning Vivek F. Farias Adam D. Jozefiak CLL 48 0 0 26 Oct 2024
Controlled Low-Rank Adaptation with Subspace Regularization for Continued Training on Large Language Models Yuheng Lu Bingshuo Qian Caixia Yuan Huixing Jiang Xiaojie Wang CLL 34 0 0 22 Oct 2024
Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation Junhong Wu Yang Zhao Yangyifan Xu Bing Liu Chengqing Zong CLL 40 1 0 17 Oct 2024
Is Parameter Collision Hindering Continual Learning in LLMs? Shuo Yang Kun-Peng Ning Yu-Yang Liu Jia-Yu Yao Yong-Hong Tian Yi-Bing Song Li Yuan MoMe CLL 24 3 0 14 Oct 2024
Synthetic Knowledge Ingestion: Towards Knowledge Refinement and Injection for Enhancing Large Language Models Jiaxin Zhang Wendi Cui Yiran Huang Kamalika Das Sricharan Kumar KELM SyDa 27 2 0 12 Oct 2024
ModalPrompt:Dual-Modality Guided Prompt for Continual Learning of Large Multimodal Models Fanhu Zeng Fei Zhu Haiyang Guo Xu-Yao Zhang Cheng-Lin Liu VLM CLL 35 8 0 08 Oct 2024
Recent Advances of Multimodal Continual Learning: A Comprehensive Survey Dianzhi Yu Xinni Zhang Yankai Chen Aiwei Liu Yifei Zhang Philip S. Yu Irwin King VLM CLL 44 9 0 07 Oct 2024
Towards LifeSpan Cognitive Systems Yu Wang Chi Han Tongtong Wu Xiaoxin He Wangchunshu Zhou ... Zexue He Wei Wang Gholamreza Haffari Heng Ji Julian McAuley KELM CLL 170 1 0 20 Sep 2024
Knowledge Adaptation Network for Few-Shot Class-Incremental Learning Ye Wang Yaxiong Wang Guoshuai Zhao Xueming Qian CLL 43 1 0 18 Sep 2024
Propulsion: Steering LLM with Tiny Fine-Tuning Md. Kowsher Nusrat Jahan Prottasha Prakash Bhat 51 4 0 17 Sep 2024
Synthetic continued pretraining Zitong Yang Neil Band Shuangping Li Emmanuel Candès Tatsunori Hashimoto CLL SyDa 41 11 0 11 Sep 2024
OneEdit: A Neural-Symbolic Collaboratively Knowledge Editing System Ningyu Zhang Zekun Xi Yujie Luo Peng Wang Bozhong Tian ... Lei Liang Qing Cui Xiaowei Zhu Jun Zhou Huajun Chen KELM 50 6 0 09 Sep 2024
Towards General Industrial Intelligence: A Survey on IIoT-Enhanced Continual Large Models Jiao Chen Jiayi He Fangfang Chen Zuohong Lv Jianhua Tang Weihua Li Zuozhu Liu Howard H. Yang Guangjie Han AI4CE 36 1 0 02 Sep 2024
On-Device Language Models: A Comprehensive Review Jiajun Xu Zhiyuan Li Wei Chen Qun Wang Xin Gao Qi Cai Ziyuan Ling 47 27 0 26 Aug 2024
Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications Qianqian Xie Dong Li Mengxi Xiao Zihao Jiang Ruoyu Xiang ... Benyou Wang Alejandro Lopez-Lira Qianqian Xie Sophia Ananiadou Junichi Tsujii AIFin AI4TS 35 15 0 20 Aug 2024
Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement Le Yu Bowen Yu Haiyang Yu Fei Huang Yongbin Li MoMe 35 5 0 06 Aug 2024
Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions Guangzhi Xiong Qiao Jin Xiao Wang Minjia Zhang Zhiyong Lu Aidong Zhang RALM 54 24 0 01 Aug 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 80 1 0 30 Jul 2024
iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvement Aoyu Pang Maonan Wang Man-On Pun Chung Shue Chen Xi Xiong 51 9 0 08 Jul 2024
UCIL: An Unsupervised Class Incremental Learning Approach for Sound Event Detection Yang Xiao Rohan Kumar Das 50 3 0 04 Jul 2024
LEMoE: Advanced Mixture of Experts Adaptor for Lifelong Model Editing of Large Language Models Renzhi Wang Piji Li KELM CLL 52 7 0 28 Jun 2024
Unlocking Continual Learning Abilities in Language Models Wenyu Du Shuang Cheng Tongxu Luo Zihan Qiu Zeyu Huang Ka Chun Cheung Reynold Cheng Jie Fu KELM CLL 51 6 0 25 Jun 2024
Finding Task-specific Subnetworks in Multi-task Spoken Language Understanding Model Hayato Futami Siddhant Arora Yosuke Kashiwagi E. Tsunoo Shinji Watanabe 39 0 0 18 Jun 2024