v1v2 (latest)

Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal

2 March 2024

Leyang Cui

Linfeng Song

Papers citing "Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal"

21 / 21 papers shown

Title
Mitigating Memorization in LLMs using Activation Steering Manan Suri Nishit Anand Amisha Bhaskar LLMSV 115 2 0 08 Mar 2025
GSCE: A Prompt Framework with Enhanced Reasoning for Reliable LLM-driven Drone Control Wenhao Wang Yanyan Li Long Jiao Jiawei Yuan 136 3 0 18 Feb 2025
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning Gangwei Jiang Caigao Jiang Zhaoyi Li Siqiao Xue Jun-ping Zhou Linqi Song Defu Lian Yin Wei CLL MU 153 2 0 16 Feb 2025
Aligning Generalisation Between Humans and Machines Filip Ilievski Barbara Hammer F. V. Harmelen Benjamin Paassen S. Saralajew ... Vered Shwartz Gabriella Skitalinskaya Clemens Stachl Gido M. van de Ven T. Villmann 286 1 0 23 Nov 2024
How Much Can We Forget about Data Contamination? Sebastian Bordt Suraj Srinivas Valentyn Boreiko U. V. Luxburg 133 2 0 04 Oct 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 149 5 0 30 Jul 2024
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning Yun Luo Zhen Yang Fandong Meng Yafu Li Jie Zhou Yue Zhang CLL KELM 190 318 0 17 Aug 2023
Progressive Prompts: Continual Learning for Language Models Anastasia Razdaibiedina Yuning Mao Rui Hou Madian Khabsa M. Lewis Amjad Almahairi VLM KELM CLL 106 142 0 29 Jan 2023
Two-stage LLM Fine-tuning with Less Specialization and More Generalization Yihan Wang Si Si Daliang Li Michal Lukasik Felix X. Yu Cho-Jui Hsieh Inderjit S Dhillon Sanjiv Kumar 131 30 0 01 Nov 2022
Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approach Prashant Shivaram Bhat Bahram Zonooz Elahe Arani SSL CLL 66 14 0 13 Jul 2022
Fine-tuned Language Models are Continual Learners Thomas Scialom Tuhin Chakrabarty Smaranda Muresan CLL LRM 191 122 0 24 May 2022
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks Yizhong Wang Swaroop Mishra Pegah Alipoormolabashi Yeganeh Kordi Amirreza Mirzaei ... Chitta Baral Yejin Choi Noah A. Smith Hannaneh Hajishirzi Daniel Khashabi ELM 123 859 0 16 Apr 2022
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning James Smith Yen-Chang Hsu John C. Balloch Yilin Shen Hongxia Jin Z. Kira CLL 106 168 0 17 Jun 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 502 10,526 0 17 Jun 2021
SimCSE: Simple Contrastive Learning of Sentence Embeddings Tianyu Gao Xingcheng Yao Danqi Chen AILaw SSL 280 3,415 0 18 Apr 2021
Continual Learning for Text Classification with Information Disentanglement Based Regularization Yufan Huang Yanzhe Zhang Jiaao Chen Xuezhi Wang Diyi Yang CLL 64 113 0 12 Apr 2021
CPR: Classifier-Projection Regularization for Continual Learning Sungmin Cha Hsiang Hsu Taebaek Hwang Flavio du Pin Calmon Taesup Moon CLL 71 77 0 12 Jun 2020
Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion Hongxu Yin Pavlo Molchanov Zhizhong Li J. Álvarez Arun Mallya Derek Hoiem N. Jha Jan Kautz 84 569 0 18 Dec 2019
Experience Replay for Continual Learning David Rolnick Arun Ahuja Jonathan Richard Schwarz Timothy Lillicrap Greg Wayne CLL 116 1,171 0 28 Nov 2018
Gradient Episodic Memory for Continual Learning David Lopez-Paz MarcÁurelio Ranzato VLM CLL 131 2,743 0 26 Jun 2017
Overcoming catastrophic forgetting in neural networks J. Kirkpatrick Razvan Pascanu Neil C. Rabinowitz J. Veness Guillaume Desjardins ... A. Grabska-Barwinska Demis Hassabis Claudia Clopath D. Kumaran R. Hadsell CLL 374 7,572 0 02 Dec 2016