Towards a Unified View of Parameter-Efficient Transfer Learning

8 October 2021

Junxian He

Chunting Zhou

Xuezhe Ma

Taylor Berg-Kirkpatrick

Graham Neubig

AAML

ArXiv PDF HTML

Papers citing "Towards a Unified View of Parameter-Efficient Transfer Learning"

50 / 632 papers shown

Title
Fine-Tuning Language Models with Differential Privacy through Adaptive Noise Allocation Xianzhi Li Ran Zmigrod Zhiqiang Ma Xiaomo Liu Xiaodan Zhu 18 1 0 03 Oct 2024
Reward-RAG: Enhancing RAG with Reward Driven Supervision Thang Nguyen Peter Chin Yu-Wing Tai RALM 42 4 0 03 Oct 2024
Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization Xinhao Yao Hongjin Qian Xiaolin Hu Gengze Xu Wei Liu Jian Luan Bin Wang Yong-Jin Liu 48 0 0 03 Oct 2024
Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts Minh Le Chau Nguyen Huy Nguyen Quyen Tran Trung Le Nhat Ho 44 4 0 03 Oct 2024
Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint? Xi Chen Kaituo Feng Changsheng Li Xunhao Lai Xiangyu Yue Ye Yuan Guoren Wang 41 7 0 02 Oct 2024
Cross-lingual Back-Parsing: Utterance Synthesis from Meaning Representation for Zero-Resource Semantic Parsing Deokhyung Kang Seonjeong Hwang Yunsu Kim Gary Geunbae Lee 31 0 0 01 Oct 2024
Learning Attentional Mixture of LoRAs for Language Model Continual Learning Jialin Liu Jianhua Wu Jie Liu Yutai Duan KELM CLL MoMe 27 1 0 29 Sep 2024
PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning Qibin Wang Xiaolin Hu Weikai Xu Wei Liu Jian Luan Bin Wang 28 1 0 25 Sep 2024
Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely Siyun Zhao Yuqing Yang Zilong Wang Zhiyuan He Luna Qiu Lili Qiu SyDa RALM 3DV 44 35 0 23 Sep 2024
Obliviate: Neutralizing Task-agnostic Backdoors within the Parameter-efficient Fine-tuning Paradigm Jaehan Kim Minkyoo Song S. Na Seungwon Shin AAML 35 0 0 21 Sep 2024
Multiple-Exit Tuning: Towards Inference-Efficient Adaptation for Vision Transformer Zheng Liu Jinchao Zhu Nannan Li Gao Huang 27 0 0 21 Sep 2024
Frequency-Guided Spatial Adaptation for Camouflaged Object Detection Shizhou Zhang Dexuan Kong Yinghui Xing Yue Lu Lingyan Ran Guoqiang Liang Hexu Wang Yanning Zhang 38 5 0 19 Sep 2024
LPT++: Efficient Training on Mixture of Long-tailed Experts Bowen Dong Pan Zhou W. Zuo VLM 42 0 0 17 Sep 2024
Propulsion: Steering LLM with Tiny Fine-Tuning Md. Kowsher Nusrat Jahan Prottasha Prakash Bhat 48 4 0 17 Sep 2024
SimMAT: Exploring Transferability from Vision Foundation Models to Any Image Modality Chenyang Lei Liyi Chen Jun Cen Xiao Chen Zhen Lei Felix Heide Ziwei Liu Qifeng Chen Zhaoxiang Zhang 47 0 0 12 Sep 2024
TempMe: Video Temporal Token Merging for Efficient Text-Video Retrieval Leqi Shen Tianxiang Hao Tao He Sicheng Zhao Pengzhang Liu Yongjun Bao Guiguang Ding Guiguang Ding 135 7 0 02 Sep 2024
Novel-WD: Exploring acquisition of Novel World Knowledge in LLMs Using Prefix-Tuning Maxime Méloux Christophe Cerisara KELM CLL 26 0 0 30 Aug 2024
Step-by-Step Unmasking for Parameter-Efficient Fine-tuning of Large Language Models Aradhye Agarwal Suhas K Ramesh Ayan Sengupta Tanmoy Chakraborty 25 1 0 26 Aug 2024
SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models Yang Cao 42 2 0 21 Aug 2024
Prompt Your Brain: Scaffold Prompt Tuning for Efficient Adaptation of fMRI Pre-trained Model Zijian Dong Yilei Wu Zijiao Chen Yichi Zhang Yueming Jin Juan Helen Zhou 36 1 0 20 Aug 2024
Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution Yucheng Ruan Xiang Lan Jingying Ma Yizhi Dong Kai He Mengling Feng LMTD 65 3 0 20 Aug 2024
Parameter-Efficient Transfer Learning under Federated Learning for Automatic Speech Recognition Xuan Kan Yonghui Xiao Tien-Ju Yang Nanxin Chen Rajiv Mathews FedML 28 0 0 19 Aug 2024
Customizing Language Models with Instance-wise LoRA for Sequential Recommendation Xiaoyu Kong Jiancan Wu An Zhang Leheng Sheng Hui Lin Xiang Wang Xiangnan He AI4TS 53 5 0 19 Aug 2024
NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Models Cheng Lin Lujun Li Dezhi Li Jie Zou Wei Xue Yike Guo AI4TS 40 4 0 18 Aug 2024
Combo: Co-speech holistic 3D human motion generation and efficient customizable adaptation in harmony Chao Xu Mingze Sun Zhi-Qi Cheng Fei-Yue Wang Yang Liu Baigui Sun Ruqi Huang Alexander G. Hauptmann VGen 45 2 0 18 Aug 2024
Snuffy: Efficient Whole Slide Image Classifier Hossein Jafarinia Alireza Alipanah Danial Hamdi Saeed Razavi Nahal Mirzaie M. Rohban 3DH 50 1 0 15 Aug 2024
CROME: Cross-Modal Adapters for Efficient Multimodal LLM Sayna Ebrahimi Sercan Ö. Arik Tejas Nama Tomas Pfister 44 1 0 13 Aug 2024
Efficient and Versatile Robust Fine-Tuning of Zero-shot Models Sungyeon Kim Boseung Jeong Donghyun Kim Suha Kwak VLM 36 2 0 11 Aug 2024
FE-Adapter: Adapting Image-based Emotion Classifiers to Videos Shreyank N. Gowda Boyan Gao David A. Clifton 32 5 0 05 Aug 2024
Memorization Capacity for Additive Fine-Tuning with Small ReLU Networks Jy-yong Sohn Dohyun Kwon Seoyeon An Kangwook Lee 46 0 0 01 Aug 2024
Task-Adapter: Task-specific Adaptation of Image Models for Few-shot Action Recognition Congqi Cao Guibiao Liao Yating Yu Kanglin Liu Lingtong Min Yanning Zhang 37 4 0 01 Aug 2024
Parameter-Efficient Fine-Tuning via Circular Convolution Aochuan Chen Jiashun Cheng Zijing Liu Ziqi Gao Fugee Tsung Yu Li Jia Li 56 2 0 27 Jul 2024
Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment Yongxin Huang Kexin Wang Goran Glavavs Iryna Gurevych 46 0 0 20 Jul 2024
Continual Learning for Remote Physiological Measurement: Minimize Forgetting and Simplify Inference Qian Liang Yan Chen Yang Hu CLL 50 2 0 19 Jul 2024
Adapt PointFormer: 3D Point Cloud Analysis via Adapting 2D Visual Transformers Mengke Li Da Li Guoqing Yang Yiu-ming Cheung Hui Huang 3DPC 48 2 0 18 Jul 2024
Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Models Xihe Qiu Haoyu Wang Xiaoyu Tan Chao Qu Yujie Xiong Yuan Cheng Yinghui Xu Wei Chu Yuan Qi LLMAG LM&Ro 43 0 0 17 Jul 2024
Enhancing Parameter Efficiency and Generalization in Large-Scale Models: A Regularized and Masked Low-Rank Adaptation Approach Yuzhu Mao Siqi Ping Zihao Zhao Yang Liu Wenbo Ding 37 1 0 16 Jul 2024
SDPT: Synchronous Dual Prompt Tuning for Fusion-based Visual-Language Pre-trained Models Yang Zhou Yongjian Wu Jiya Saiyin Bingzheng Wei Maode Lai Eric Chang Yan Xu VLM 46 0 0 16 Jul 2024
Image Compression for Machine and Human Vision with Spatial-Frequency Adaptation Han Li Shaohui Li Shuangrui Ding Wenrui Dai Maida Cao Chenglin Li Junni Zou Hongkai Xiong VLM 40 5 0 13 Jul 2024
Seq-to-Final: A Benchmark for Tuning from Sequential Distributions to a Final Time Point Christina X. Ji Ahmed M. Alaa David Sontag OOD 52 0 0 12 Jul 2024
Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software Dezhi Ran Mengzhou Wu Wei Yang Tao Xie AI4CE 36 1 0 11 Jul 2024
SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning Haiwen Diao Bo Wan Xu Jia Yunzhi Zhuge Ying Zhang Huchuan Lu Long Chen VLM 50 4 0 10 Jul 2024
Parameter Efficient Fine Tuning for Multi-scanner PET to PET Reconstruction Yumin Kim Gayoon Choi Seong Jae Hwang 37 0 0 10 Jul 2024
Learn and Don't Forget: Adding a New Language to ASR Foundation Models Mengjie Qian Siyuan Tang Rao Ma Kate Knill Mark J. F. Gales CLL 39 3 0 09 Jul 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 54 24 0 08 Jul 2024
See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition Chongjie Si Xiaokang Yang Wei Shen 42 5 0 07 Jul 2024
SBoRA: Low-Rank Adaptation with Regional Weight Updates L. Po Yuyang Liu Haoxuan Wu Tianqi Zhang W. Yu Zeyu Jiang Kun Li 38 1 0 07 Jul 2024
HiDe-PET: Continual Learning via Hierarchical Decomposition of Parameter-Efficient Tuning Liyuan Wang Jingyi Xie Xingxing Zhang Hang Su Jun Zhu CLL 47 5 0 07 Jul 2024
Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models Yuyan Chen Qiang Fu Ge Fan Lun Du Jian-Guang Lou Shi Han Dongmei Zhang Zhixu Li Yanghua Xiao MoE 29 16 0 04 Jul 2024
LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models Hayder Elesedy Pedro M. Esperança Silviu Vlad Oprea Mete Ozay KELM 36 2 0 03 Jul 2024