Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterization

6 May 2023

Anastasia Razdaibiedina

Madian Khabsa

Jimmy Ba

Papers citing "Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterization"

36 / 36 papers shown

Title
PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning Zongqian Li Yixuan Su Nigel Collier MoE 21 0 0 14 May 2025
Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients Yezhen Wang Zhouhao Yang Brian K Chen Fanyi Pu Bo-wen Li Tianyu Gao Kenji Kawaguchi 41 0 0 03 May 2025
Generalized Tensor-based Parameter-Efficient Fine-Tuning via Lie Group Transformations Chongjie Si Zhiyi Shi Xuehui Wang Yichen Xiao Xiaokang Yang Wei-Ming Shen AI4CE 65 0 0 01 Apr 2025
Advancing Prompt-Based Methods for Replay-Independent General Continual Learning Zhiqi Kang Liyuan Wang Xingxing Zhang Karteek Alahari CLL VLM 40 1 0 02 Mar 2025
PaCA: Partial Connection Adaptation for Efficient Fine-Tuning Sunghyeon Woo Sol Namkung Sunwoo Lee Inho Jeong Beomseok Kim Dongsuk Jeon 39 0 0 28 Feb 2025
Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation Reza Akbarian Bafghi Carden Bagwell Avinash Ravichandran Ashish Shrivastava M. Raissi 48 0 0 28 Jan 2025
MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba Masakazu Yoshimura Teruaki Hayashi Yota Maeda Mamba 100 2 0 06 Nov 2024
DEAN: Deactivating the Coupled Neurons to Mitigate Fairness-Privacy Conflicts in Large Language Models Chen Qian Dongrui Liu Jie Zhang Yong Liu Jing Shao 37 1 0 22 Oct 2024
BiDoRA: Bi-level Optimization-Based Weight-Decomposed Low-Rank Adaptation Peijia Qin Ruiyi Zhang Pengtao Xie 31 1 0 13 Oct 2024
ALLoRA: Adaptive Learning Rate Mitigates LoRA Fatal Flaws Hai Huang Randall Balestriero 35 0 0 13 Oct 2024
Parameter-Efficient Fine-Tuning of Large Language Models using Semantic Knowledge Tuning Nusrat Jahan Prottasha Asif Mahmud Md. Shohanur Islam Sobuj Prakash Bhat Md. Kowsher Niloofar Yousefi O. Garibay 32 4 0 11 Oct 2024
Propulsion: Steering LLM with Tiny Fine-Tuning Md. Kowsher Nusrat Jahan Prottasha Prakash Bhat 48 4 0 17 Sep 2024
See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition Chongjie Si Xiaokang Yang Wei Shen 42 5 0 07 Jul 2024
SBoRA: Low-Rank Adaptation with Regional Weight Updates L. Po Yuyang Liu Haoxuan Wu Tianqi Zhang W. Yu Zeyu Jiang Kun Li 38 1 0 07 Jul 2024
LoRA-GA: Low-Rank Adaptation with Gradient Approximation Shaowen Wang Linxi Yu Jian Li ALM AI4CE 26 29 0 06 Jul 2024
PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning Jiaru Zou Mengyu Zhou Tao Li Shi Han Dongmei Zhang 51 6 0 02 Jul 2024
Fighting Randomness with Randomness: Mitigating Optimisation Instability of Fine-Tuning using Delayed Ensemble and Noisy Interpolation Branislav Pecher Ján Cegin Róbert Belanec Jakub Simko Ivan Srba M. Bieliková 44 1 0 18 Jun 2024
SuperPos-Prompt: Enhancing Soft Prompt Tuning of Language Models with Superposition of Multi Token Embeddings MohammadAli SadraeiJavaeri Ehsaneddin Asgari A. Mchardy Hamid R. Rabiee VLM AAML 26 0 0 07 Jun 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou Shuaiwen Leon Song Jianlong Wu Liqiang Nie Guohao Li 45 6 0 07 Jun 2024
Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuning Chongjie Si Xuehui Wang Xue Yang Zhengqin Xu Qingyun Li Jifeng Dai Yu Qiao Xiaokang Yang Wei Shen 31 8 0 23 May 2024
Efficient Prompt Tuning by Multi-Space Projection and Prompt Fusion Pengxiang Lan Enneng Yang Yuting Liu Guibing Guo Linying Jiang Jianzhe Zhao Xingwei Wang VLM AAML 38 1 0 19 May 2024
StablePT: Towards Stable Prompting for Few-shot Learning via Input Separation Xiaoming Liu Chen Liu Zhaohan Zhang Chengzhengxu Li Longtian Wang Y. Lan Chao Shen VLM 49 4 0 30 Apr 2024
LayerNorm: A key component in parameter-efficient fine-tuning Taha ValizadehAslani Hualou Liang 51 1 0 29 Mar 2024
DoRA: Weight-Decomposed Low-Rank Adaptation Shih-yang Liu Chien-Yi Wang Hongxu Yin Pavlo Molchanov Yu-Chiang Frank Wang Kwang-Ting Cheng Min-Hung Chen 32 342 0 14 Feb 2024
Soft Prompt Tuning for Cross-Lingual Transfer: When Less is More Fred Philippy Siwen Guo Shohreh Haddadan Cedric Lothritz Jacques Klein Tegawende F. Bissyande AAML VLM 22 1 0 06 Feb 2024
Extending Whisper with prompt tuning to target-speaker ASR Hao Ma Zhiyuan Peng Mingjie Shao Jing Li Ju Liu VLM 32 12 0 13 Dec 2023
Non-Intrusive Adaptation: Input-Centric Parameter-efficient Fine-Tuning for Versatile Multimodal Modeling Yaqing Wang Jialin Wu T. Dabral Jiageng Zhang Geoff Brown ... Frederick Liu Yi Liang Bo Pang Michael Bendersky Radu Soricut VLM 25 14 0 18 Oct 2023
Decomposed Prompt Tuning via Low-Rank Reparameterization Yao Xiao Lu Xu Jiaxi Li Wei Lu Xiaoli Li VLM 22 6 0 16 Oct 2023
PRE: Vision-Language Prompt Learning with Reparameterization Encoder Anh Pham Thi Minh An Duc Nguyen Georgios Tzimiropoulos VPVLM VLM 25 3 0 14 Sep 2023
InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding Junda Wu Tong Yu Rui Wang Zhao-quan Song Ruiyi Zhang Handong Zhao Chaochao Lu Shuai Li Ricardo Henao VLM 33 23 0 08 Jun 2023
Prompting Language-Informed Distribution for Compositional Zero-Shot Learning Wentao Bao Lichang Chen Heng-Chiao Huang Yu Kong CoGe VLM 29 12 0 23 May 2023
Representation Projection Invariance Mitigates Representation Collapse Anastasia Razdaibiedina A. Khetan Zohar Karnin Daniel Khashabi Vishaal Kapoor V. Madan 34 5 0 23 May 2022
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu Brian Lester Noah Constant Rami Al-Rfou Daniel Cer VLM LRM 137 277 0 15 Oct 2021
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 238 806 0 14 Oct 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 261 4,489 0 23 Jan 2020