Towards a Unified View of Parameter-Efficient Transfer Learning

8 October 2021

Junxian He

Chunting Zhou

Xuezhe Ma

Taylor Berg-Kirkpatrick

Graham Neubig

AAML

ArXiv PDF HTML

Papers citing "Towards a Unified View of Parameter-Efficient Transfer Learning"

50 / 633 papers shown

Title
Dual3D: Efficient and Consistent Text-to-3D Generation with Dual-mode Multi-view Latent Diffusion Xinyang Li Zhangyu Lai Linning Xu Jianfei Guo Liujuan Cao Shengchuan Zhang Bo Dai Rongrong Ji DiffM 53 8 0 16 May 2024
Striking a Balance between Classical and Deep Learning Approaches in Natural Language Processing Pedagogy Aditya Joshi Jake Renzella Pushpak Bhattacharyya Saurav Jha Xiangyu Zhang 19 0 0 16 May 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation Jie Xu Karthikeyan P. Saravanan Rogier van Dalen Haaris Mehmood David Tuckey Mete Ozay 56 5 0 10 May 2024
Group-aware Parameter-efficient Updating for Content-Adaptive Neural Video Compression Zhenghao Chen Luping Zhou Zhihao Hu Dong Xu 28 9 0 07 May 2024
FedSC: Provable Federated Self-supervised Learning with Spectral Contrastive Objective over Non-i.i.d. Data Shusen Jing Anlan Yu Shuai Zhang Songyang Zhang FedML 31 1 0 07 May 2024
Parameter-Efficient Fine-Tuning with Discrete Fourier Transform Ziqi Gao Qichao Wang Aochuan Chen Zijing Liu Bingzhe Wu Liang Chen Jia Li 43 25 0 05 May 2024
Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning Jing Xu Jingzhao Zhang 39 7 0 04 May 2024
A Survey of Time Series Foundation Models: Generalizing Time Series Representation with Large Language Model Weiqi Zhang Jiexia Ye Ke Yi Yongzi Yu Ziyue Li Jia Li Fugee Tsung AI4TS AI4CE 45 22 0 03 May 2024
RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarization Dongqi Pu Vera Demberg 38 5 0 01 May 2024
BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models Jiamin Li Le Xu Hong-Yu Xu Aditya Akella 24 1 0 28 Apr 2024
Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approaches Clément Christophe Praveen K Kanithi Prateek Munjal Tathagata Raha Nasir Hayat ... Charles Chen Natalia Vassilieva Boulbaba Ben Amor Marco AF Pimentel Shadab Khan AI4MH LM&MA 33 28 0 23 Apr 2024
ColA: Collaborative Adaptation with Gradient Learning Enmao Diao Qi Le Suya Wu Xinran Wang Ali Anwar Jie Ding Vahid Tarokh 35 1 0 22 Apr 2024
MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts Dengchun Li Yingzi Ma Naizheng Wang Zhengmao Ye Zhiyuan Cheng ... Yan Zhang Lei Duan Jie Zuo Cal Yang Mingjie Tang MoE 40 44 0 22 Apr 2024
MergeNet: Knowledge Migration across Heterogeneous Models, Tasks, and Modalities Kunxi Li Tianyu Zhan Kairui Fu Shengyu Zhang Kun Kuang Jiwei Li Zhou Zhao Fei Wu MoMe 24 0 0 20 Apr 2024
AdaIR: Exploiting Underlying Similarities of Image Restoration Tasks with Adapters Hao-Wei Chen Yu-Syuan Xu Kelvin C. K. Chan Hsien-Kai Kuo Chun-Yi Lee Ming-Hsuan Yang 31 1 0 17 Apr 2024
LoRA Dropout as a Sparsity Regularizer for Overfitting Control Yang Lin Xinyu Ma Xu Chu Yujie Jin Zhibang Yang Yasha Wang Hong-yan Mei 52 19 0 15 Apr 2024
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 46 7 0 13 Apr 2024
Improving Continuous Sign Language Recognition with Adapted Image Models Lianyu Hu Tongkai Shi Liqing Gao Zekang Liu Wei Feng VLM 26 5 0 12 Apr 2024
PromptSync: Bridging Domain Gaps in Vision-Language Models through Class-Aware Prototype Alignment and Discrimination Anant Khandelwal VLM 23 1 0 11 Apr 2024
Certified PEFTSmoothing: Parameter-Efficient Fine-Tuning with Randomized Smoothing Chengyan Fu Wenjie Wang AAML 32 0 0 08 Apr 2024
DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model Chao Gao Sai Qian Zhang ALM 123 7 0 08 Apr 2024
Facial Affective Behavior Analysis with Instruction Tuning Yifan Li Anh Dao Wentao Bao Zhen Tan Tianlong Chen Huan Liu Yu Kong CVBM 60 15 0 07 Apr 2024
Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation Tong Su Xin Peng Sarubi Thillainathan David Guzmán Surangika Ranathunga En-Shiun Annie Lee 37 2 0 05 Apr 2024
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation Xinyu Ma Xu Chu Zhibang Yang Yang Lin Xin Gao Junfeng Zhao 46 7 0 05 Apr 2024
ReFT: Representation Finetuning for Language Models Zhengxuan Wu Aryaman Arora Zheng Wang Atticus Geiger Daniel Jurafsky Christopher D. Manning Christopher Potts OffRL 38 1 0 04 Apr 2024
Personalized LLM Response Generation with Parameterized Memory Injection Kai Zhang Lizhi Qing Yangyang Kang 36 11 0 04 Apr 2024
BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Models Qi Luo Hengxu Yu Xiao Li 47 1 0 03 Apr 2024
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models Fanxu Meng Zhaohui Wang Muhan Zhang VLM 64 73 0 03 Apr 2024
Mixture-of-Depths: Dynamically allocating compute in transformer-based language models David Raposo Sam Ritter Blake A. Richards Timothy Lillicrap Peter C. Humphreys Adam Santoro MoE 40 69 0 02 Apr 2024
Pre-trained Vision and Language Transformers Are Few-Shot Incremental Learners Keon-Hee Park Kyungwoo Song Gyeong-Moon Park VLM CLL 40 16 0 02 Apr 2024
AAdaM at SemEval-2024 Task 1: Augmentation and Adaptation for Multilingual Semantic Textual Relatedness Miaoran Zhang Mingyang Wang Jesujoba Oluwadara Alabi Dietrich Klakow VLM 46 4 0 01 Apr 2024
MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learning Ahmed A. Agiza Marina Neseem Sherief Reda MoE 48 12 0 29 Mar 2024
Model Stock: All we need is just a few fine-tuned models Dong-Hwan Jang Sangdoo Yun Dongyoon Han OODD MoMe 34 38 0 28 Mar 2024
Dual-Personalizing Adapter for Federated Foundation Models Yiyuan Yang Guodong Long Taoshu Shen Jing Jiang Michael Blumenstein 94 19 0 28 Mar 2024
Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning Huiyi Wang Haodong Lu Lina Yao Dong Gong KELM CLL 45 8 0 27 Mar 2024
PE: A Poincare Explanation Method for Fast Text Hierarchy Generation Qian Chen Dongyang Li Xiaofeng He Hongzhao Li Hongyu Yi 24 0 0 25 Mar 2024
DeepMachining: Online Prediction of Machining Errors of Lathe Machines Xiang-Li Lu Hwai-Jung Hsu Che-Wei Chou H. T. Kung Chen-Hsin Lee Sheng-Mao Cheng AI4CE 31 0 0 25 Mar 2024
ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Models Zequan Liu Jiawen Lyn Wei-wei Zhu Xing Tian Yvette Graham OffRL 37 10 0 24 Mar 2024
Not All Attention is Needed: Parameter and Computation Efficient Transfer Learning for Multi-modal Large Language Models Qiong Wu Weihao Ye Yiyi Zhou Xiaoshuai Sun Rongrong Ji MoE 49 1 0 22 Mar 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 150 310 0 21 Mar 2024
Improving LoRA in Privacy-preserving Federated Learning Youbang Sun Zitao Li Yaliang Li Bolin Ding 35 57 0 18 Mar 2024
SuperLoRA: Parameter-Efficient Unified Adaptation of Multi-Layer Attention Modules Xiangyu Chen Jing Liu Ye Wang Pu Wang Matthew Brand Guanghui Wang T. Koike-Akino 48 7 0 18 Mar 2024
OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning Lingyi Hong Shilin Yan Renrui Zhang Wanyun Li Xinyu Zhou ... Kaixun Jiang Yiting Chen Jinglun Li Zhaoyu Chen Wenqiang Zhang VLM 34 38 0 14 Mar 2024
PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Traps Ruixuan Liu Tianhao Wang Yang Cao Li Xiong AAML SILM 56 15 0 14 Mar 2024
Introducing Routing Functions to Vision-Language Parameter-Efficient Fine-Tuning with Low-Rank Bottlenecks Tingyu Qu Tinne Tuytelaars Marie-Francine Moens MoE 46 2 0 14 Mar 2024
An Empirical Study of Parameter Efficient Fine-tuning on Vision-Language Pre-train Model Yuxin Tian Mouxing Yang Yunfan Li Dayiheng Liu Xingzhang Ren Xiaocui Peng Jiancheng Lv VLM 42 0 0 13 Mar 2024
Matrix-Transformation Based Low-Rank Adaptation (MTLoRA): A Brain-Inspired Method for Parameter-Efficient Fine-Tuning Yao Liang Yuwei Wang Yang Li Yi Zeng 44 0 0 12 Mar 2024
SPA: Towards A Computational Friendly Cloud-Base and On-Devices Collaboration Seq2seq Personalized Generation Yanming Liu Xinyue Peng Jiannan Cao Le Dai Xingzu Liu Mingbang Wang Weihao Liu SyDa 44 2 0 11 Mar 2024
Controllable Prompt Tuning For Balancing Group Distributional Robustness Hoang Phan Andrew Gordon Wilson Qi Lei 43 5 0 05 Mar 2024
Derivative-Free Optimization for Low-Rank Adaptation in Large Language Models Feihu Jin Yin Liu Ying Tan 35 3 0 04 Mar 2024