Scaling Laws for Forgetting When Fine-Tuning Large Language Models

11 January 2024

Papers citing "Scaling Laws for Forgetting When Fine-Tuning Large Language Models"

7 / 7 papers shown

Title
A Domain-Agnostic Scalable AI Safety Ensuring Framework Beomjun Kim Kangyeon Kim Sunwoo Kim Heejin Ahn 57 0 0 29 Apr 2025
Human-inspired Perspectives: A Survey on AI Long-term Memory Zihong He Weizhe Lin Hao Zheng Fan Zhang Matt Jones Laurence Aitchison X. Xu Miao Liu Per Ola Kristensson Junxiao Shen 77 2 0 01 Nov 2024
MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning Hanqing Wang Zeguan Xiao Shuo Wang Guanhua Chen Guanhua Chen 44 19 0 13 Jun 2024
FlashBack:Efficient Retrieval-Augmented Language Modeling for Long Context Inference Runheng Liu Xingchen Xiao Heyan Huang Zewen Chi Zhijing Wu RALM KELM 34 0 0 07 May 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 363 12,003 0 04 Mar 2022
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 264 4,489 0 23 Jan 2020
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models Cheolhyoung Lee Kyunghyun Cho Wanmo Kang MoE 249 205 0 25 Sep 2019