Towards a Unified View of Parameter-Efficient Transfer Learning

8 October 2021

Junxian He

Chunting Zhou

Xuezhe Ma

Taylor Berg-Kirkpatrick

Graham Neubig

AAML

ArXiv PDF HTML

Papers citing "Towards a Unified View of Parameter-Efficient Transfer Learning"

50 / 633 papers shown

Title
Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis Xin Zhou Dingkang Liang Wei Xu Xingkui Zhu Yihan Xu Zhikang Zou Xiang Bai 34 26 0 03 Mar 2024
STAR: Constraint LoRA with Dynamic Active Learning for Data-Efficient Fine-Tuning of Large Language Models Linhai Zhang Jialong Wu Deyu Zhou Guoqiang Xu 30 4 0 02 Mar 2024
Bias Mitigation in Fine-tuning Pre-trained Models for Enhanced Fairness and Efficiency Yixuan Zhang Feng Zhou 21 3 0 01 Mar 2024
Efficient Adapter Tuning of Pre-trained Speech Models for Automatic Speaker Verification Mufan Sang John H. L. Hansen 41 6 0 01 Mar 2024
AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Merging Yiran Zhao Wenxuan Zhang Huiming Wang Kenji Kawaguchi Lidong Bing MoMe 32 15 0 29 Feb 2024
FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning Xupeng Miao Gabriele Oliaro Xinhao Cheng Vineeth Kada Ruohan Gao ... April Yang Yingcheng Wang Mengdi Wu Colin Unger Zhihao Jia MoE 94 9 0 29 Feb 2024
ShapeLLM: Universal 3D Object Understanding for Embodied Interaction Zekun Qi Runpei Dong Shaochen Zhang Haoran Geng Chunrui Han Zheng Ge Li Yi Kaisheng Ma 41 51 0 27 Feb 2024
When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method Biao Zhang Zhongtao Liu Colin Cherry Orhan Firat LRM 63 126 0 27 Feb 2024
ConSept: Continual Semantic Segmentation via Adapter-based Vision Transformer Bowen Dong Guanglei Yang W. Zuo Lei Zhang 54 1 0 26 Feb 2024
Advancing Parameter Efficiency in Fine-tuning via Representation Editing Muling Wu Wenhao Liu Xiaohua Wang Tianlong Li Changze Lv Zixuan Ling Jianhao Zhu Cenyuan Zhang Xiaoqing Zheng Xuanjing Huang 22 19 0 23 Feb 2024
PEMT: Multi-Task Correlation Guided Mixture-of-Experts Enables Parameter-Efficient Transfer Learning Zhisheng Lin Han Fu Chenghao Liu Zhuo Li Jianling Sun MoE MoMe 33 5 0 23 Feb 2024
Reliable LLM-based User Simulator for Task-Oriented Dialogue Systems Ivan Sekulić Silvia Terragni Victor Guimaraes Nghia Khau Bruna Guedes Modestas Filipavicius A. Manso Roland Mathis 46 5 0 20 Feb 2024
The Hidden Space of Transformer Language Adapters Jesujoba Oluwadara Alabi Marius Mosbach Matan Eyal Dietrich Klakow Mor Geva 56 7 1 20 Feb 2024
MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Models Tongxu Luo Jiahe Lei Fangyu Lei Weihao Liu Shizhu He Jun Zhao Kang Liu MoE ALM 35 18 0 20 Feb 2024
A Comprehensive Review of Machine Learning Advances on Data Change: A Cross-Field Perspective Jeng-Lin Li Chih-Fan Hsu Ming-Ching Chang Wei-Chao Chen OOD 44 2 0 20 Feb 2024
SIBO: A Simple Booster for Parameter-Efficient Fine-Tuning Zhihao Wen Jie Zhang Yuan Fang MoE 34 3 0 19 Feb 2024
InfuserKI: Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integration Fali Wang Runxue Bao Suhang Wang Wenchao Yu Yanchi Liu Wei Cheng Haifeng Chen KELM 21 8 0 18 Feb 2024
When LLMs Meets Acoustic Landmarks: An Efficient Approach to Integrate Speech into Large Language Models for Depression Detection Xiangyu Zhang Hexin Liu Kaishuai Xu Qiquan Zhang Daijiao Liu Beena Ahmed Julien Epps 23 7 0 17 Feb 2024
QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning Hossein Rajabzadeh Mojtaba Valipour Tianshu Zhu Marzieh S. Tahaei Hyock Ju Kwon Ali Ghodsi Boxing Chen Mehdi Rezagholizadeh 32 9 0 16 Feb 2024
Recovering the Pre-Fine-Tuning Weights of Generative Models Eliahu Horwitz Jonathan Kahana Yedid Hoshen 50 9 0 15 Feb 2024
Quantized Embedding Vectors for Controllable Diffusion Language Models Cheng Kang Xinye Chen Yong Hu Daniel Novak 31 0 0 15 Feb 2024
DoRA: Weight-Decomposed Low-Rank Adaptation Shih-yang Liu Chien-Yi Wang Hongxu Yin Pavlo Molchanov Yu-Chiang Frank Wang Kwang-Ting Cheng Min-Hung Chen 32 342 0 14 Feb 2024
BBox-Adapter: Lightweight Adapting for Black-Box Large Language Models Haotian Sun Yuchen Zhuang Wei Wei Chao Zhang Bo Dai 18 3 0 13 Feb 2024
Empowering Federated Learning for Massive Models with NVIDIA FLARE Holger R. Roth Ziyue Xu Yuan-Ting Hsieh Adithya Renduchintala Isaac Yang ... Camir Ricketts Daguang Xu Chester Chen Yan Cheng Andrew Feng AI4CE 26 5 0 12 Feb 2024
LiFi: Lightweight Controlled Text Generation with Fine-Grained Control Codes Chufan Shi Deng Cai Yujiu Yang 50 3 0 10 Feb 2024
Learning to Route Among Specialized Experts for Zero-Shot Generalization Mohammed Muqeeth Haokun Liu Yufan Liu Colin Raffel MoMe 37 34 0 08 Feb 2024
Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes Lucio Dery Steven Kolawole Jean-Francois Kagey Virginia Smith Graham Neubig Ameet Talwalkar 44 28 0 08 Feb 2024
Read to Play (R2-Play): Decision Transformer with Multimodal Game Instruction Yonggang Jin Ge Zhang Hao Zhao Tianyu Zheng Jiawei Guo Liuyu Xiang Shawn Yue Stephen W. Huang Zhaofeng He Jie Fu OffRL 31 4 0 06 Feb 2024
Time-, Memory- and Parameter-Efficient Visual Adaptation Otniel-Bogdan Mercea Alexey Gritsenko Cordelia Schmid Anurag Arnab VLM 35 13 0 05 Feb 2024
KS-Lottery: Finding Certified Lottery Tickets for Multilingual Language Models Fei Yuan Chang Ma Shuai Yuan Qiushi Sun Lei Li 39 3 0 05 Feb 2024
On Catastrophic Inheritance of Large Foundation Models Hao Chen Bhiksha Raj Xing Xie Jindong Wang AI4CE 56 12 0 02 Feb 2024
LoTR: Low Tensor Rank Weight Adaptation Daniel Bershatsky Daria Cherniuk Talgat Daulbaev A. Mikhalev Ivan V. Oseledets 57 7 0 02 Feb 2024
How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation? Rheeya Uppaal Yixuan Li Junjie Hu 37 4 0 31 Jan 2024
X-PEFT: eXtremely Parameter-Efficient Fine-Tuning for Extreme Multi-Profile Scenarios Namju Kwak Taesup Kim MoE 23 0 0 29 Jan 2024
MM-LLMs: Recent Advances in MultiModal Large Language Models Duzhen Zhang Yahan Yu Jiahua Dong Chenxing Li Dan Su Chenhui Chu Dong Yu OffRL LRM 52 179 0 24 Jan 2024
Consistency Guided Knowledge Retrieval and Denoising in LLMs for Zero-shot Document-level Relation Triplet Extraction Qi Sun Kun Huang Xiaocui Yang Rong Tong Kun Zhang Soujanya Poria 34 18 0 24 Jan 2024
APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference Bowen Zhao Hannaneh Hajishirzi Qingqing Cao 29 17 0 22 Jan 2024
PRILoRA: Pruned and Rank-Increasing Low-Rank Adaptation Nadav Benedek Lior Wolf 32 5 0 20 Jan 2024
Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models Zhengxin Zhang Dan Zhao Xupeng Miao Gabriele Oliaro Qing Li Yong-jia Jiang Zhihao Jia MQ 46 7 0 13 Jan 2024
Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion Ruilin Luo Tianle Gu Haoling Li Junzhe Li Zicheng Lin Jiayi Li Yujiu Yang AI4CE 36 7 0 11 Jan 2024
RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation Mahdi Nikdan Soroush Tabesh Elvir Crnčević Dan Alistarh 10 27 0 09 Jan 2024
Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning Wenhan Xia Chengwei Qin Elad Hazan 60 52 0 08 Jan 2024
Towards Integrated Fine-tuning and Inference when Generative AI meets Edge Intelligence Ning Chen Zhipeng Cheng Xuwei Fan Xiaoyu Xia Lianfen Huang 56 1 0 05 Jan 2024
BA-SAM: Scalable Bias-Mode Attention Mask for Segment Anything Model Yiran Song Qianyu Zhou Hefei Ling Deng-Ping Fan Xuequan Lu Lizhuang Ma VLM 38 14 0 04 Jan 2024
PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learning Haiyang Guo Fei Zhu Wenzhuo Liu Xu-Yao Zhang Cheng-Lin Liu CLL 35 6 0 04 Jan 2024
Cheetah: Natural Language Generation for 517 African Languages Ife Adebara AbdelRahim Elmadany Muhammad Abdul-Mageed 29 4 0 02 Jan 2024
Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models Terry Yue Zhuo A. Zebaze Nitchakarn Suppattarachai Leandro von Werra H. D. Vries Qian Liu Niklas Muennighoff ALM 41 15 0 01 Jan 2024
Overview of the PromptCBLUE Shared Task in CHIP2023 Wei-wei Zhu Xiaoling Wang Mosha Chen Buzhou Tang LM&MA ELM 39 7 0 29 Dec 2023
AdapterDistillation: Non-Destructive Task Composition with Knowledge Distillation Junjie Wang Yicheng Chen Wangshu Zhang Sen Hu Teng Xu Jing Zheng VLM 34 1 0 26 Dec 2023
A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Software Engineering Tasks Wentao Zou Qi Li Jidong Ge Chuanyi Li Xiaoyu Shen LiGuo Huang Bin Luo 29 5 0 25 Dec 2023