Towards a Unified View of Parameter-Efficient Transfer Learning

8 October 2021

Junxian He

Chunting Zhou

Xuezhe Ma

Taylor Berg-Kirkpatrick

Graham Neubig

AAML

ArXiv PDF HTML

Papers citing "Towards a Unified View of Parameter-Efficient Transfer Learning"

50 / 633 papers shown

Title
Decomposed Prompt Tuning via Low-Rank Reparameterization Yao Xiao Lu Xu Jiaxi Li Wei Lu Xiaoli Li VLM 25 6 0 16 Oct 2023
Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models? Yu-Lin Tsai Chia-Yi Hsu Chulin Xie Chih-Hsun Lin Jia-You Chen Bo-wen Li Pin-Yu Chen Chia-Mu Yu Chun-ying Huang DiffM 44 78 0 16 Oct 2023
LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models Yixiao Li Yifan Yu Chen Liang Pengcheng He Nikos Karampatziakis Weizhu Chen Tuo Zhao MQ 41 125 0 12 Oct 2023
An Empirical Study of Instruction-tuning Large Language Models in Chinese Q. Si Tong Wang Zheng Lin Xu Zhang Yanan Cao Weiping Wang ALM 74 16 0 11 Oct 2023
TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models Zuxin Liu Jesse Zhang Kavosh Asadi Yao Liu Ding Zhao Shoham Sabach Rasool Fakoor ALM AI4CE 30 26 0 09 Oct 2023
Hierarchical Side-Tuning for Vision Transformers Weifeng Lin Ziheng Wu Wentao Yang Mingxin Huang Jun Huang Lianwen Jin 31 8 0 09 Oct 2023
Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation Md Kaykobad Reza Ashley Prater-Bennette Ulugbek S. Kamilov 28 5 0 06 Oct 2023
Prompting and Adapter Tuning for Self-supervised Encoder-Decoder Speech Model Kai-Wei Chang Ming-Hsin Chen Yun-Ping Lin Jing Neng Hsu Paul Kuo-Ming Huang Chien-yu Huang Shang-Wen Li Hung-yi Lee 25 6 0 04 Oct 2023
Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models Yiwen Tang Ivan Tang Ray Gu Dong Wang Eric Zhang Bin Zhao Xuelong Li 3DPC 34 20 0 04 Oct 2023
Selective Feature Adapter for Dense Vision Transformers XueQing Deng Qi Fan Xiaojie Jin Linjie Yang Peng Wang 37 0 0 03 Oct 2023
ScaLearn: Simple and Highly Parameter-Efficient Task Transfer by Learning to Scale Markus Frohmann Carolin Holtermann Shahed Masoudian Anne Lauscher Navid Rekabsaz 47 2 0 02 Oct 2023
PETA: Parameter-Efficient Trojan Attacks Lauren Hong Ting Wang AAML 50 1 0 01 Oct 2023
Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering Han Zhou Xingchen Wan Lev Proleev Diana Mincu Jilin Chen Katherine A. Heller Subhrajit Roy UQLM 31 54 0 29 Sep 2023
Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks Hao Chen Jindong Wang Ankit Shah Ran Tao Hongxin Wei Berfin cSimcsek Masashi Sugiyama Bhiksha Raj 44 26 0 29 Sep 2023
PC-Adapter: Topology-Aware Adapter for Efficient Domain Adaption on Point Clouds with Rectified Pseudo-label Joonhyung Park Hyunjin Seo Eunho Yang 3DPC 24 10 0 29 Sep 2023
BLIP-Adapter: Parameter-Efficient Transfer Learning for Mobile Screenshot Captioning Ching-Yu Chiang I-Hua Chang Shih-Wei Liao 55 1 0 26 Sep 2023
QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models Yuhui Xu Lingxi Xie Xiaotao Gu Xin Chen Heng Chang Hengheng Zhang Zhensu Chen Xiaopeng Zhang Qi Tian MQ 21 92 0 26 Sep 2023
Sparsely Shared LoRA on Whisper for Child Speech Recognition W. Liu Ying Qin Zhiyuan Peng Tan Lee 27 17 0 21 Sep 2023
OWL: A Large Language Model for IT Operations Hongcheng Guo Jian Yang Jiaheng Liu Liqun Yang Linzheng Chai ... Tieqiao Zheng Liangfan Zheng Bo Zhang Ke Xu Zhoujun Li VLM 66 41 0 17 Sep 2023
MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation Cheng Chen Juzheng Miao Dufan Wu Zhiling Yan Sekeun Kim ... Lichao Sun Xiang Li Tianming Liu Pheng-Ann Heng Quanzheng Li MedIm 55 58 0 16 Sep 2023
Scaled Prompt-Tuning for Few-Shot Natural Language Generation Ting Hu Christoph Meinel Haojin Yang 16 0 0 13 Sep 2023
The first step is the hardest: Pitfalls of Representing and Tokenizing Temporal Data for Large Language Models Dimitris Spathis F. Kawsar AI4TS 36 18 0 12 Sep 2023
Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Review Pengzhou Cheng Zongru Wu Wei Du Haodong Zhao Wei Lu Gongshen Liu SILM AAML 37 18 0 12 Sep 2023
MultiWay-Adapater: Adapting large-scale multi-modal models for scalable image-text retrieval Zijun Long George Killick R. McCreadie Gerardo Aragon Camarasa 14 4 0 04 Sep 2023
Parameter and Computation Efficient Transfer Learning for Vision-Language Pre-trained Models Qiong Wu Wei Yu Yiyi Zhou Shubin Huang Xiaoshuai Sun Rongrong Ji VLM 26 7 0 04 Sep 2023
MerA: Merging Pretrained Adapters For Few-Shot Learning Shwai He Run-Ze Fan Liang Ding Li Shen Dinesh Manocha Dacheng Tao MoMe 42 11 0 30 Aug 2023
UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memory Haiwen Diao Bo Wan Wenjie Qu Xuecong Jia Huchuan Lu Long Chen VLM 39 18 0 28 Aug 2023
FwdLLM: Efficient FedLLM using Forward Gradient Mengwei Xu Dongqi Cai Yaozong Wu Xiang Li Shangguang Wang FedML 57 25 0 26 Aug 2023
Overcoming Generic Knowledge Loss with Selective Parameter Update Wenxuan Zhang Paul Janson Rahaf Aljundi Mohamed Elhoseiny KELM CLL 39 10 0 23 Aug 2023
IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning Feiyu F. Zhang Liangzhi Li Jun-Cheng Chen Zhouqian Jiang Bowen Wang Yiming Qian 51 33 0 23 Aug 2023
MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation Jinpeng Wang Ziyun Zeng Yunxiao Wang Yuting Wang Xingyu Lu Tianxiang Li Jun Yuan Rui Zhang Haitao Zheng Shutao Xia 44 43 0 22 Aug 2023
LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning Jun Lu Lei Yu Xiaojia Li Li Yang Chun Zuo ALM 27 73 0 22 Aug 2023
VLN-PETL: Parameter-Efficient Transfer Learning for Vision-and-Language Navigation Yanyuan Qiao Zheng Yu Qi Wu VLM 22 16 0 20 Aug 2023
VL-PET: Vision-and-Language Parameter-Efficient Tuning via Granularity Control Zi-Yuan Hu Yanyang Li M. Lyu Liwei Wang VLM 35 15 0 18 Aug 2023
SimDA: Simple Diffusion Adapter for Efficient Video Generation Zhen Xing Qi Dai Hang-Rui Hu Zuxuan Wu Yu-Gang Jiang VGen DiffM 40 81 0 18 Aug 2023
FedPerfix: Towards Partial Model Personalization of Vision Transformers in Federated Learning Guangyu Sun Matías Mendieta Jun Luo Shandong Wu Chong Chen 38 16 0 17 Aug 2023
Challenges and Opportunities of Using Transformer-Based Multi-Task Learning in NLP Through ML Lifecycle: A Survey Lovre Torbarina Tin Ferkovic Lukasz Roguski Velimir Mihelčić Bruno Šarlija Z. Kraljevic 31 5 0 16 Aug 2023
Separate the Wheat from the Chaff: Model Deficiency Unlearning via Parameter-Efficient Module Operation Xinshuo Hu Dongfang Li Baotian Hu Zihao Zheng Zhenyu Liu Hao Fei KELM MU 40 26 0 16 Aug 2023
Platypus: Quick, Cheap, and Powerful Refinement of LLMs Ariel N. Lee Cole J. Hunter Nataniel Ruiz ALM ObjD 34 136 0 14 Aug 2023
SLoRA: Federated Parameter Efficient Fine-Tuning of Language Models Sara Babakniya A. Elkordy Yahya H. Ezzeldin Qingfeng Liu Kee-Bong Song Mostafa El-Khamy Salman Avestimehr 27 57 0 12 Aug 2023
Towards Instance-adaptive Inference for Federated Learning Chunhui Feng Kai Yu Nian Liu Xinxing Xu Salman Khan W. Zuo FedML 39 12 0 11 Aug 2023
Adapter-based Selective Knowledge Distillation for Federated Multi-domain Meeting Summarization Xiachong Feng Xiaocheng Feng Xiyuan Du MingSung Kan Bing Qin FedML 29 5 0 07 Aug 2023
SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning Keyu Duan Qian Liu Tat-Seng Chua Shuicheng Yan Wei Tsang Ooi Qizhe Xie Junxian He 35 57 0 03 Aug 2023
Low-Parameter Federated Learning with Large Language Models Jing Jiang Xiangyang Liu Chenyou Fan 33 24 0 26 Jul 2023
LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition Chengsong Huang Qian Liu Bill Yuchen Lin Tianyu Pang Chao Du Min Lin MoMe 38 186 0 25 Jul 2023
Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community Qingyao Ai Ting Bai Bo Zhao Yi-Ju Chang Jiawei Chen ... Peng Zhang Fan Zhang Wei-na Zhang Hao Fei Xiaofei Zhu 52 59 0 19 Jul 2023
A Comprehensive Overview of Large Language Models Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Ajmal Mian OffRL 70 538 0 12 Jul 2023
Towards Robust and Efficient Continual Language Learning Adam Fisch Amal Rannen-Triki Razvan Pascanu J. Bornschein Angeliki Lazaridou E. Gribovskaya MarcÁurelio Ranzato CLL 37 1 0 11 Jul 2023
Answering Ambiguous Questions via Iterative Prompting Weiwei Sun Hengyi Cai Hongshen Chen Pengjie Ren Zhumin Chen Maarten de Rijke Zhaochun Ren 45 9 0 08 Jul 2023
OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models Shengding Hu Ning Ding Weilin Zhao Xingtai Lv Zhen Zhang Zhiyuan Liu Maosong Sun 37 7 0 05 Jul 2023