SAPT: A Shared Attention Framework for Parameter-Efficient Continual
Learning of Large Language Models

SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models

16 January 2024

Wanxiang Che

Papers citing "SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models"

16 / 16 papers shown

Title
SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning Jinpeng Chen Runmin Cong Yuzhi Zhao Hongzheng Yang Guangneng Hu H. Ip Sam Kwong CLL KELM 83 0 0 05 May 2025
Think Small, Act Big: Primitive Prompt Learning for Lifelong Robot Manipulation Yuanqi Yao Siao Liu Haoming Song Delin Qu Qizhi Chen Yan Ding Bin Zhao Zhilin Wang X. Li Dong Wang CLL 66 0 0 01 Apr 2025
Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities Weixiang Zhao Xingyu Sui Jiahe Guo Yulin Hu Yang Deng Yanyan Zhao Bing Qin Wanxiang Che Tat-Seng Chua Ting Liu ELM LRM 59 4 0 23 Mar 2025
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning Gangwei Jiang Caigao Jiang Zhaoyi Li Siqiao Xue Jun-ping Zhou Linqi Song Defu Lian Yin Wei CLL MU 63 1 0 16 Feb 2025
Unlocking Continual Learning Abilities in Language Models Wenyu Du Shuang Cheng Tongxu Luo Zihan Qiu Zeyu Huang Ka Chun Cheung Reynold Cheng Jie Fu KELM CLL 48 6 0 25 Jun 2024
Towards Incremental Learning in Large Language Models: A Critical Review M. Jovanovic Peter Voss ELM CLL KELM 37 5 0 28 Apr 2024
Continual Learning of Large Language Models: A Comprehensive Survey Haizhou Shi Zihao Xu Hengyi Wang Weiyi Qin Wenyuan Wang Yibin Wang Zifeng Wang Sayna Ebrahimi Hao Wang CLL KELM LRM 52 64 0 25 Apr 2024
Continual Learning for Large Language Models: A Survey Tongtong Wu Linhao Luo Yuan-Fang Li Shirui Pan Thuy-Trang Vu Gholamreza Haffari CLL LRM KELM 28 102 0 02 Feb 2024
Orthogonal Subspace Learning for Language Model Continual Learning Xiao Wang Tianze Chen Qiming Ge Han Xia Rong Bao Rui Zheng Qi Zhang Tao Gui Xuanjing Huang CLL 122 89 0 22 Oct 2023
Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation Chengwei Qin Junwon Lee Shafiq R. Joty CLL 43 12 0 15 Oct 2023
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 130 100 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 319 11,953 0 04 Mar 2022
LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5 Chengwei Qin Shafiq R. Joty CLL 178 98 0 14 Oct 2021
Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions Mohammad Aliannejadi Julia Kiseleva A. Chuklin Jeffrey Stephen Dalton Mikhail Burtsev 73 96 0 13 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018