Towards a Unified View of Parameter-Efficient Transfer Learning

8 October 2021

Junxian He

Chunting Zhou

Xuezhe Ma

Taylor Berg-Kirkpatrick

Graham Neubig

AAML

ArXiv PDF HTML

Papers citing "Towards a Unified View of Parameter-Efficient Transfer Learning"

50 / 633 papers shown

Title
LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models Hayder Elesedy Pedro M. Esperança Silviu Vlad Oprea Mete Ozay KELM 36 2 0 03 Jul 2024
Soft Language Prompts for Language Transfer Ivan Vykopal Simon Ostermann Marian Simko AAML 42 1 0 02 Jul 2024
PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning Jiaru Zou Mengyu Zhou Tao Li Shi Han Dongmei Zhang 51 6 0 02 Jul 2024
Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models Zihan Wang Deli Chen Damai Dai Runxin Xu Zhuoshu Li Y. Wu MoE ALM 40 2 0 02 Jul 2024
Increasing Model Capacity for Free: A Simple Strategy for Parameter Efficient Fine-tuning Haobo Song Hao Zhao Soumajit Majumder Tao Lin 30 3 0 01 Jul 2024
FAFE: Immune Complex Modeling with Geodesic Distance Loss on Noisy Group Frames R. Wu Ruihan Guo Rui Wang Shitong Luo Yue Xu Jiahan Li Jianzhu Ma Qiang Liu Yunan Luo Jian Peng 39 1 0 01 Jul 2024
Dataset Size Recovery from LoRA Weights Mohammad Salama Jonathan Kahana Eliahu Horwitz Yedid Hoshen 42 5 0 27 Jun 2024
Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning Arijit Sehanobish Avinava Dubey Krzysztof Choromanski Somnath Basu Roy Chowdhury Deepali Jain Vikas Sindhwani Snigdha Chaturvedi ALM 43 1 0 25 Jun 2024
BlockLLM: Memory-Efficient Adaptation of LLMs by Selecting and Optimizing the Right Coordinate Blocks A. Ramesh Vignesh Ganapathiraman I. Laradji Mark W. Schmidt 38 1 0 25 Jun 2024
MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearning Jiali Cheng Hadi Amiri BDL 43 3 0 21 Jun 2024
Bayesian-LoRA: LoRA based Parameter Efficient Fine-Tuning using Optimal Quantization levels and Rank Values trough Differentiable Bayesian Gates Cristian Meo Ksenia Sycheva Anirudh Goyal Justin Dauwels MQ 24 4 0 18 Jun 2024
From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP Marius Mosbach Vagrant Gautam Tomás Vergara-Browne Dietrich Klakow Mor Geva AI4CE 38 6 0 18 Jun 2024
Mixture-of-Subspaces in Low-Rank Adaptation Taiqiang Wu Jiahao Wang Zhe Zhao Ngai Wong 49 21 0 16 Jun 2024
ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation Yurun Song Junchen Zhao Ian G. Harris S. Jyothi 32 3 0 16 Jun 2024
Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Efforts Zhaoxuan Tan Zheyuan Liu Meng Jiang 36 20 0 15 Jun 2024
Reflecting on the State of Rehearsal-free Continual Learning with Pretrained Models Lukas Thede Karsten Roth Olivier J. Hénaff Matthias Bethge Zeynep Akata CLL 47 5 0 13 Jun 2024
Dual-Pipeline with Low-Rank Adaptation for New Language Integration in Multilingual ASR Yerbolat Khassanov Zhipeng Chen Tianfeng Chen Tze Yuang Chong Wei Li Jun Zhang Lu Lu Yuxuan Wang AI4CE 18 0 0 12 Jun 2024
SE/BN Adapter: Parametric Efficient Domain Adaptation for Speaker Recognition Tianhao Wang Lantian Li D. Wang 35 0 0 12 Jun 2024
Adapters Strike Back Jan-Martin O. Steitz Stefan Roth 27 5 0 10 Jun 2024
Multi-Prompting Decoder Helps Better Language Understanding Zifeng Cheng Zhaoling Chen Zhiwei Jiang Yafeng Yin Shiping Ge Yuliang Liu Qing Gu AI4CE 50 1 0 10 Jun 2024
Learning Fine-Grained Controllability on Speech Generation via Efficient Fine-Tuning Chung-Ming Chien Andros Tjandra Apoorv Vyas Matt Le Bowen Shi Wei-Ning Hsu 32 0 0 10 Jun 2024
An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models Xiongtao Zhou Jie He Yuhua Ke Guangyao Zhu Víctor Gutiérrez-Basulto Jeff Z. Pan 40 11 0 07 Jun 2024
MEFT: Memory-Efficient Fine-Tuning through Sparse Adapter Jitai Hao Weiwei Sun Xin Xin Qi Meng Zhumin Chen Pengjie Ren Zhaochun Ren MoE 42 2 0 07 Jun 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou Shuaiwen Leon Song Jianlong Wu Liqiang Nie Guohao Li 45 6 0 07 Jun 2024
Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe Alicja Ziarko Albert Q. Jiang Bartosz Piotrowski Wenda Li M. Jamnik Piotr Miłoś 34 0 0 06 Jun 2024
Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision Minglei Li Peng Ye Yongqi Huang Lin Zhang Tao Chen Tong He Jiayuan Fan Wanli Ouyang MoE 34 4 0 05 Jun 2024
Low-Rank Adaption on Transformer-based Oriented Object Detector for Satellite Onboard Processing of Remote Sensing Images Xinyang Pu Feng Xu 32 3 0 04 Jun 2024
FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language Models Tao Fan Guoqiang Ma Yan Kang Hanlin Gu Yuanfeng Song Lixin Fan Kai Chen Qiang Yang 26 9 0 04 Jun 2024
OLoRA: Orthonormal Low-Rank Adaptation of Large Language Models Kerim Büyükakyüz AI4CE 21 5 0 03 Jun 2024
Few-Shot Classification of Interactive Activities of Daily Living (InteractADL) Zane Durante Robathan Harries Edward Vendrow Zelun Luo Yuta Kyuragi Kazuki Kozuka Fei-Fei Li Ehsan Adeli VLM 25 0 0 03 Jun 2024
LoFiT: Localized Fine-tuning on LLM Representations Fangcong Yin Xi Ye Greg Durrett 38 13 0 03 Jun 2024
MACT: Model-Agnostic Cross-Lingual Training for Discourse Representation Structure Parsing Jiangming Liu 37 1 0 03 Jun 2024
QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptation Zhuo Chen Rumen Dangovski Charlotte Loh Owen Dugan Di Luo Marin Soljacic MQ 30 8 0 31 May 2024
RAP: Efficient Text-Video Retrieval with Sparse-and-Correlated Adapter Meng Cao Haoran Tang Jinfa Huang Peng Jin Can Zhang Ruyang Liu Long Chen Xiaodan Liang Li-ming Yuan Ge Li 101 11 0 29 May 2024
Low-Rank Few-Shot Adaptation of Vision-Language Models Maxime Zanella Ismail Ben Ayed OffRL VLM 56 26 0 28 May 2024
OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning Pengxiang Li Lu Yin Xiaowei Gao Shiwei Liu 31 7 0 28 May 2024
Semantic are Beacons: A Semantic Perspective for Unveiling Parameter-Efficient Fine-Tuning in Knowledge Learning Renzhi Wang Piji Li 37 4 0 28 May 2024
IAPT: Instruction-Aware Prompt Tuning for Large Language Models Wei-wei Zhu Aaron Xuxiang Tian Congrui Yin Yuan Ni Xiaoling Wang Guotong Xie 48 0 0 28 May 2024
DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution Yulong Mao Kaiyu Huang Changhao Guan Ganglin Bao Fengran Mo Jinan Xu 29 10 0 27 May 2024
Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective Akiyoshi Tomihari Issei Sato 30 4 0 27 May 2024
NuwaTS: a Foundation Model Mending Every Incomplete Time Series Jinguo Cheng Chunwei Yang Wanlin Cai Yuxuan Liang Yuankai Wu AI4TS 42 1 0 24 May 2024
Prompt Tuning Strikes Back: Customizing Foundation Models with Low-Rank Prompt Adaptation Abhinav C. P. Jain Swarat Chaudhuri Thomas W. Reps Christopher M. Jermaine 22 1 0 24 May 2024
VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks Yang Li Shaobo Han Shihao Ji MoE 27 11 0 24 May 2024
CLIP model is an Efficient Online Lifelong Learner Leyuan Wang Liuyu Xiang Yujie Wei Yunlong Wang Zhaofeng He VLM CLL 32 3 0 24 May 2024
Recurrent Early Exits for Federated Learning with Heterogeneous Clients Royson Lee Javier Fernandez-Marques S. Hu Da Li Stefanos Laskaridis L. Dudziak Timothy M. Hospedales Ferenc Huszár Nicholas D. Lane 36 3 0 23 May 2024
Federated Domain-Specific Knowledge Transfer on Large Language Models Using Synthetic Data Haoran Li Xinyuan Zhao Dadi Guo Hanlin Gu Ziqian Zeng Yuxing Han Yangqiu Song Lixin Fan Qiang Yang 23 1 0 23 May 2024
Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuning Chongjie Si Xuehui Wang Xue Yang Zhengqin Xu Qingyun Li Jifeng Dai Yu Qiao Xiaokang Yang Wei Shen 31 8 0 23 May 2024
Multi-domain Knowledge Graph Collaborative Pre-training and Prompt Tuning for Diverse Downstream Tasks Yichi Zhang Binbin Hu Zhuo Chen Lingbing Guo Ziqi Liu Qing Cui Lei Liang Hua-zeng Chen Wen Zhang CLL 35 1 0 21 May 2024
Layout Agnostic Human Activity Recognition in Smart Homes through Textual Descriptions Of Sensor Triggers (TDOST) Megha Thukral Sourish Gunesh Dhekane S. Hiremath H. Haresamudram Thomas Ploetz 43 3 0 20 May 2024
Efficient Prompt Tuning by Multi-Space Projection and Prompt Fusion Pengxiang Lan Enneng Yang Yuting Liu Guibing Guo Linying Jiang Jianzhe Zhao Xingwei Wang VLM AAML 38 1 0 19 May 2024