Towards a Unified View of Parameter-Efficient Transfer Learning

8 October 2021

Junxian He

Chunting Zhou

Xuezhe Ma

Taylor Berg-Kirkpatrick

Graham Neubig

AAML

ArXiv PDF HTML

Papers citing "Towards a Unified View of Parameter-Efficient Transfer Learning"

50 / 632 papers shown

Title
A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models Qinggang Zhang Shengyuan Chen Yuanchen Bei Zheng Yuan Huachi Zhou Zijin Hong Junnan Dong Hao-Heng Chen Yi-Ju Chang Xiao Huang 3DV 70 7 0 21 Jan 2025
SAM-DA: Decoder Adapter for Efficient Medical Domain Adaptation Javier Gamazo Tejero Moritz Schmid Pablo Márquez-Neila M. Zinkernagel Sebastian Wolf Raphael Sznitman MedIm 44 0 0 12 Jan 2025
An Efficient Adaptive Compression Method for Human Perception and Machine Vision Tasks Lei Liu Zhenghao Chen Zhihao Hu Dong Xu 36 4 0 08 Jan 2025
RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation Jun Liu Zhenglun Kong Peiyan Dong Changdi Yang Xuan Shen ... Wei Niu Wenbin Zhang Xue Lin Dong Huang Yanzhi Wang ALM 43 2 0 08 Jan 2025
IDEAL: Leveraging Infinite and Dynamic Characterizations of Large Language Models for Query-focused Summarization Jie Cao Dian Jiao Qiang Yan Wenqiao Zhang Siliang Tang Yueting Zhuang 44 1 0 08 Jan 2025
CM3T: Framework for Efficient Multimodal Learning for Inhomogeneous Interaction Datasets Tanay Agrawal Mohammed Guermal Michal Balazia François Brémond 29 0 0 08 Jan 2025
DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models Xiaolin Hu Xiang Cheng Peiyu Liu Wei Liu Jian Luan Bin Wang Y. Liu 42 1 0 31 Dec 2024
Learning to Adapt to Low-Resource Paraphrase Generation Zhigen Li Yanmeng Wang Rizhao Fan Ye Wang Jianfeng Li Shaojun Wang 121 3 0 22 Dec 2024
Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Prediction Sepideh Maleki Jan-Christian Huetter Kangway V Chuang Gabriele Scalia Tommaso Biancalani Tommaso Biancalani AI4CE 90 2 0 18 Dec 2024
Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes Aodi Li Liansheng Zhuang Xiao Long Minghong Yao Shafei Wang 186 0 0 18 Dec 2024
When Vision Models Meet Parameter Efficient Look-Aside Adapters Without Large-Scale Audio Pretraining Juan Yeo Jinkwan Jang Kyubyung Chae Seongkyu Mun Taesup Kim VLM 62 0 0 08 Dec 2024
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models Fan Wang Juyong Jiang Chansung Park Sunghun Kim Jing Tang 94 1 0 08 Dec 2024
Adapter-based Approaches to Knowledge-enhanced Language Models -- A Survey Alexander Fichtl Juraj Vladika Georg Groh KELM 78 0 0 25 Nov 2024
SeqProFT: Applying LoRA Finetuning for Sequence-only Protein Property Predictions Shuo Zhang Jian K. Liu 67 0 0 18 Nov 2024
LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models Runming Yang Taiqiang Wu Jiahao Wang Pengfei Hu Ngai Wong Yujiu Yang Yujiu Yang 142 0 0 11 Nov 2024
PSELDNets: Pre-trained Neural Networks on Large-scale Synthetic Datasets for Sound Event Localization and Detection Jinbo Hu Yin Cao Ming Wu Fang Kang Feiran Yang Wenwu Wang Mark D. Plumbley J. Yang 31 0 0 10 Nov 2024
MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba Masakazu Yoshimura Teruaki Hayashi Yota Maeda Mamba 100 2 0 06 Nov 2024
AM Flow: Adapters for Temporal Processing in Action Recognition Tanay Agrawal Abid Ali A. Dantcheva François Brémond 39 0 0 04 Nov 2024
Parameter-Efficient Fine-Tuning of Large Language Models for Unit Test Generation: An Empirical Study André Storhaug Jingyue Li ALM 53 1 0 04 Nov 2024
Pin-Tuning: Parameter-Efficient In-Context Tuning for Few-Shot Molecular Property Prediction Liang Wang Qiang Liu Shaozhen Liu Xin Sun Shu Wu Liang Wang 39 2 0 02 Nov 2024
IntLoRA: Integral Low-rank Adaptation of Quantized Diffusion Models Hang Guo Yawei Li Tao Dai Shu-Tao Xia Luca Benini MQ 39 1 0 29 Oct 2024
LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization Jui-Nan Yen Si Si Zhao Meng Felix X. Yu Sai Surya Duvvuri Inderjit Dhillon Cho-Jui Hsieh Sanjiv Kumar 27 3 0 27 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 49 3 0 24 Oct 2024
MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning Jingfan Zhang Yi Zhao Dan Chen Xing Tian Huanran Zheng Wei Zhu MoE 37 12 0 23 Oct 2024
ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning Zhiwei Hao Jianyuan Guo Li Shen Yong Luo Han Hu Yonggang Wen VLM 26 0 0 23 Oct 2024
Towards Real Zero-Shot Camouflaged Object Segmentation without Camouflaged Annotations Cheng Lei Jie Fan Xinran Li Tianzhu Xiang Ao Li Ce Zhu Le Zhang 28 0 0 22 Oct 2024
Towards Optimal Adapter Placement for Efficient Transfer Learning Aleksandra I. Nowak Otniel-Bogdan Mercea Anurag Arnab Jonas Pfeiffer Yann N. Dauphin Utku Evci 25 0 0 21 Oct 2024
Router-Tuning: A Simple and Effective Approach for Enabling Dynamic-Depth in Transformers Shwai He Tao Ge Guoheng Sun Bowei Tian Xiaoyang Wang Ang Li MoE 54 1 0 17 Oct 2024
Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models Sajjad Ghiasvand Yifan Yang Zhiyu Xue Mahnoosh Alizadeh Zheng Zhang Ramtin Pedarsani FedML 38 3 0 16 Oct 2024
Triple Modality Fusion: Aligning Visual, Textual, and Graph Data with Large Language Models for Multi-Behavior Recommendations Luyi Ma Xiaohan Li Zezhong Fan Kai Zhao Jianpeng Xu Praveen Kanumala Kaushiki Nag Sushant Kumar Sushant Kumar Kannan Achan 46 4 0 16 Oct 2024
Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models Kai Yao P. Gao Lichun Li Yuan Zhao Xiaofeng Wang Wei Wang Jianke Zhu 26 1 0 15 Oct 2024
Scalable Multi-Domain Adaptation of Language Models using Modular Experts Peter Schafhalter Shun Liao Yanqi Zhou Chih-Kuan Yeh Arun Kandoor James Laudon MoE 29 1 0 14 Oct 2024
Is Parameter Collision Hindering Continual Learning in LLMs? Shuo Yang Kun-Peng Ning Yu-Yang Liu Jia-Yu Yao Yong-Hong Tian Yi-Bing Song Li Yuan MoMe CLL 24 3 0 14 Oct 2024
RoCoFT: Efficient Finetuning of Large Language Models with Row-Column Updates Md. Kowsher Tara Esmaeilbeig Chun-Nam Yu Mojtaba Soltanalian Niloofar Yousefi 27 0 0 14 Oct 2024
BiDoRA: Bi-level Optimization-Based Weight-Decomposed Low-Rank Adaptation Peijia Qin Ruiyi Zhang Pengtao Xie 31 1 0 13 Oct 2024
ALLoRA: Adaptive Learning Rate Mitigates LoRA Fatal Flaws Hai Huang Randall Balestriero 35 0 0 13 Oct 2024
Towards Efficient Visual-Language Alignment of the Q-Former for Visual Reasoning Tasks Sungkyung Kim Adam Lee Junyoung Park Andrew Chung Jusang Oh Jay-Yoon Lee 26 3 0 12 Oct 2024
Zero-shot Commonsense Reasoning over Machine Imagination Hyuntae Park Yeachan Kim Jun-Hyung Park S. Lee ReLM VLM LRM 24 1 0 12 Oct 2024
MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning Yaming Yang Dilxat Muhtar Yelong Shen Yuefeng Zhan Jianfeng Liu ... Denvy Deng Feng Sun Qi Zhang Weizhu Chen Yunhai Tong MoE MoMe 43 2 0 12 Oct 2024
DA-Ada: Learning Domain-Aware Adapter for Domain Adaptive Object Detection Hao Li Rui Zhang Hantao Yao X. Zhang Yifan Hao Xinkai Song Xiaqing Li Yongwei Zhao Ling Li Yunji Chen ObjD VLM 31 3 0 11 Oct 2024
Parameter-Efficient Fine-Tuning of Large Language Models using Semantic Knowledge Tuning Nusrat Jahan Prottasha Asif Mahmud Md. Shohanur Islam Sobuj Prakash Bhat Md. Kowsher Niloofar Yousefi O. Garibay 30 4 0 11 Oct 2024
Parameter-Efficient Fine-Tuning of State Space Models Kevin Galim Wonjun Kang Yuchen Zeng H. Koo Kangwook Lee 31 4 0 11 Oct 2024
Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation Grigory Malinovsky Umberto Michieli Hasan Hammoud Taha Ceritli Hayder Elesedy Mete Ozay Peter Richtárik AI4CE 32 2 0 10 Oct 2024
SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture Jiayi Han Liang Du Hongwei Du Xiangguo Zhou Yiwen Wu Weibo Zheng Donghong Han CLL MoMe MoE 38 2 0 10 Oct 2024
QuAILoRA: Quantization-Aware Initialization for LoRA Neal Lawton Aishwarya Padmakumar Judith Gaspers Jack FitzGerald Anoop Kumar Greg Ver Steeg Aram Galstyan MQ 31 0 0 09 Oct 2024
Parameter-Efficient Fine-Tuning via Selective Discrete Cosine Transform Yixian Shen Qi Bi Jia-Hong Huang Hongyi Zhu Anuj Pathania 31 1 0 09 Oct 2024
Efficient Few-shot Learning for Multi-label Classification of Scientific Documents with Many Classes Tim Schopf Alexander Blatzheim Nektarios Machner Florian Matthes VLM 23 1 0 08 Oct 2024
Hyperbolic Fine-tuning for Large Language Models Menglin Yang Aosong Feng Bo Xiong Jihong Liu Irwin King Rex Ying 23 1 0 05 Oct 2024
LoRTA: Low Rank Tensor Adaptation of Large Language Models Ignacio Hounie Charilaos I. Kanatsoulis Arnuv Tandon Alejandro Ribeiro 36 0 0 05 Oct 2024
On Unsupervised Prompt Learning for Classification with Black-box Language Models Zhen-Yu Zhang Jiandong Zhang Huaxiu Yao Gang Niu Masashi Sugiyama 21 2 0 04 Oct 2024