Towards a Unified View of Parameter-Efficient Transfer Learning

8 October 2021

Junxian He

Chunting Zhou

Xuezhe Ma

Taylor Berg-Kirkpatrick

Graham Neubig

AAML

ArXiv PDF HTML

Papers citing "Towards a Unified View of Parameter-Efficient Transfer Learning"

50 / 633 papers shown

Title
Improving Reliability of Fine-tuning with Block-wise Optimisation Basel Barakat Qiang Huang 19 1 0 15 Jan 2023
Parameter-Efficient Fine-Tuning Design Spaces Jiaao Chen Aston Zhang Xingjian Shi Mu Li Alexander J. Smola Diyi Yang 42 59 0 04 Jan 2023
SERENGETI: Massively Multilingual Language Models for Africa Ife Adebara AbdelRahim Elmadany Muhammad Abdul-Mageed Alcides Alcoba Inciarte 24 29 0 21 Dec 2022
KronA: Parameter Efficient Tuning with Kronecker Adapter Ali Edalati Marzieh S. Tahaei I. Kobyzev V. Nia J. Clark Mehdi Rezagholizadeh 45 92 0 20 Dec 2022
HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation Hamish Ivison Akshita Bhagia Yizhong Wang Hannaneh Hajishirzi Matthew E. Peters 36 16 0 20 Dec 2022
Reasoning with Language Model Prompting: A Survey Shuofei Qiao Yixin Ou Ningyu Zhang Xiang Chen Yunzhi Yao Shumin Deng Chuanqi Tan Fei Huang Huajun Chen ReLM ELM LRM 71 311 0 19 Dec 2022
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting Zheng-Xin Yong Hailey Schoelkopf Niklas Muennighoff Alham Fikri Aji David Ifeoluwa Adelani ... Genta Indra Winata Stella Biderman Edward Raff Dragomir R. Radev Vassilina Nikoulina CLL VLM AI4CE LRM 35 81 0 19 Dec 2022
An Extensible Plug-and-Play Method for Multi-Aspect Controllable Text Generation Xuancheng Huang Zijun Liu Peng Li Tao Li Maosong Sun Yang Liu 27 10 0 19 Dec 2022
Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning? Runpei Dong Zekun Qi Linfeng Zhang Junbo Zhang Jian‐Yuan Sun Zheng Ge Li Yi Kaisheng Ma ViT 3DPC 29 84 0 16 Dec 2022
Federated Few-Shot Learning for Mobile NLP Dongqi Cai Shangguang Wang Yaozong Wu F. Lin Mengwei Xu FedML 13 12 0 12 Dec 2022
Parameter-Efficient Finetuning of Transformers for Source Code Shamil Ayupov Nadezhda Chirkova 22 17 0 12 Dec 2022
Searching for Effective Multilingual Fine-Tuning Methods: A Case Study in Summarization Yiwei Qin Graham Neubig Pengfei Liu 21 3 0 12 Dec 2022
Federated Learning for Inference at Anytime and Anywhere Zicheng Liu Da Li Javier Fernandez-Marques Stefanos Laskaridis Yan Gao L. Dudziak Stan Z. Li S. Hu Timothy M. Hospedales FedML 32 5 0 08 Dec 2022
Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learning Cheng-Hao Tu Zheda Mai Wei-Lun Chao 29 44 0 06 Dec 2022
FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer Shibo Jie Zhi-Hong Deng 23 126 0 06 Dec 2022
Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning Shih-Cheng Huang Shi Wang Min-Han Shih Saurav Sahay Hung-yi Lee 39 0 0 02 Dec 2022
CHAPTER: Exploiting Convolutional Neural Network Adapters for Self-supervised Speech Models Zih-Ching Chen Yu-Shun Sung Hung-yi Lee 29 16 0 01 Dec 2022
On the Effectiveness of Parameter-Efficient Fine-Tuning Z. Fu Haoran Yang Anthony Man-Cho So Wai Lam Lidong Bing Nigel Collier 27 156 0 28 Nov 2022
Deep representation learning: Fundamentals, Perspectives, Applications, and Open Challenges K. T. Baghaei Amirreza Payandeh Pooya Fayyazsanavi Shahram Rahimi Zhiqian Chen Somayeh Bakhtiari Ramezani FaML AI4TS 35 6 0 27 Nov 2022
VoP: Text-Video Co-operative Prompt Tuning for Cross-Modal Retrieval Siteng Huang Biao Gong Yulin Pan Jianwen Jiang Yiliang Lv Yuyuan Li Donglin Wang VLM VPVLM 22 41 0 23 Nov 2022
Structured Pruning Adapters Lukas Hedegaard Aman Alok Juby Jose Alexandros Iosifidis 38 10 0 17 Nov 2022
Prompt Tuning for Parameter-efficient Medical Image Segmentation Marc Fischer Alexander Bartler Bin Yang SSeg 22 18 0 16 Nov 2022
Parameter-Efficient Tuning on Layer Normalization for Pre-trained Language Models Wang Qi Yu-Ping Ruan Y. Zuo Taihao Li 21 18 0 16 Nov 2022
A Universal Discriminator for Zero-Shot Generalization Haike Xu Zongyu Lin Jing Zhou Yanan Zheng Zhilin Yang AI4CE 21 14 0 15 Nov 2022
FedTune: A Deep Dive into Efficient Federated Fine-Tuning with Pre-trained Transformers Jinyu Chen Wenchao Xu Song Guo Junxiao Wang Jie Zhang Yining Qi FedML 28 32 0 15 Nov 2022
FPT: Improving Prompt Tuning Efficiency via Progressive Training Yufei Huang Yujia Qin Huadong Wang Yichun Yin Maosong Sun Zhiyuan Liu Qun Liu VLM LRM 35 6 0 13 Nov 2022
COPEN: Probing Conceptual Knowledge in Pre-trained Language Models Hao Peng Xiaozhi Wang Shengding Hu Hailong Jin Lei Hou Juanzi Li Zhiyuan Liu Qun Liu 18 22 0 08 Nov 2022
Pretraining in Deep Reinforcement Learning: A Survey Zhihui Xie Zichuan Lin Junyou Li Shuai Li Deheng Ye OffRL OnRL AI4CE 26 23 0 08 Nov 2022
On the Domain Adaptation and Generalization of Pretrained Language Models: A Survey Xu Guo Han Yu LM&MA VLM 28 29 0 06 Nov 2022
Universal Deep Image Compression via Content-Adaptive Optimization with Adapters Koki Tsubota Hiroaki Akutsu Kiyoharu Aizawa 29 17 0 02 Nov 2022
Two-stage LLM Fine-tuning with Less Specialization and More Generalization Yihan Wang Si Si Daliang Li Michal Lukasik Felix X. Yu Cho-Jui Hsieh Inderjit S Dhillon Sanjiv Kumar 46 29 0 01 Nov 2022
Adapter-Based Extension of Multi-Speaker Text-to-Speech Model for New Speakers Cheng-Ping Hsieh Subhankar Ghosh Boris Ginsburg 41 18 0 01 Nov 2022
Parameter-Efficient Tuning Makes a Good Classification Head Zhuoyi Yang Ming Ding Yanhui Guo Qingsong Lv Jie Tang VLM 43 14 0 30 Oct 2022
Differentiable Data Augmentation for Contrastive Sentence Representation Learning Tianduo Wang Wei Lu SSL 32 9 0 29 Oct 2022
Parameter-efficient transfer learning of pre-trained Transformer models for speaker verification using adapters Junyi Peng Themos Stafylakis Rongzhi Gu Oldvrich Plchot Ladislav Movsner Lukávs Burget JanHonza'' vCernocký 42 22 0 28 Oct 2022
Inducer-tuning: Connecting Prefix-tuning and Adapter-tuning Yifan Chen Devamanyu Hazarika Mahdi Namazifar Yang Liu Di Jin Dilek Z. Hakkani-Tür 21 3 0 26 Oct 2022
Learning Better Intent Representations for Financial Open Intent Classification Xianzhi Li Will Aitken Xiao-Dan Zhu Stephen W. Thomas AIFin 14 8 0 25 Oct 2022
Evaluating Parameter Efficient Learning for Generation Peng-Tao Xu M. Patwary Shrimai Prabhumoye Virginia Adams R. Prenger Ming-Yu Liu Nayeon Lee M. Shoeybi Bryan Catanzaro MoE 33 3 0 25 Oct 2022
Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning Jing Yi Weize Chen Yujia Qin Yankai Lin Ning Ding Xu Han Zhiyuan Liu Maosong Sun Jie Zhou 20 2 0 24 Oct 2022
NVIDIA FLARE: Federated Learning from Simulation to Real-World H. Roth Yan Cheng Yuhong Wen Isaac Yang Ziyue Xu ... Daguang Xu Nic Ma Prerna Dogra Mona G. Flores Andrew Feng FedML AI4CE 27 96 0 24 Oct 2022
Unsupervised Non-transferable Text Classification Guangtao Zeng Wei Lu 32 6 0 23 Oct 2022
Meta-learning Pathologies from Radiology Reports using Variance Aware Prototypical Networks Arijit Sehanobish Kawshik Kannan Nabila Abraham Anasuya Das Benjamin Odry VLM 26 0 0 22 Oct 2022
CPL: Counterfactual Prompt Learning for Vision and Language Models Xuehai He Diji Yang Weixi Feng Tsu-jui Fu Arjun Reddy Akula Varun Jampani P. Narayana Sugato Basu William Yang Wang Qing Guo VPVLM VLM 50 15 0 19 Oct 2022
Continued Pretraining for Better Zero- and Few-Shot Promptability Zhaofeng Wu IV RobertL.Logan Pete Walsh Akshita Bhagia Dirk Groeneveld Sameer Singh Iz Beltagy VLM 41 12 0 19 Oct 2022
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning Shuo Xie Jiahao Qiu Ankita Pasad Li Du Qing Qu Hongyuan Mei 35 16 0 18 Oct 2022
Tiny-Attention Adapter: Contexts Are More Important Than the Number of Parameters Hongyu Zhao Hao Tan Hongyuan Mei MoE 36 16 0 18 Oct 2022
Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning Tianxiang Sun Zhengfu He Qinen Zhu Xipeng Qiu Xuanjing Huang VLM VPVLM 17 20 0 14 Oct 2022
DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation Mojtaba Valipour Mehdi Rezagholizadeh I. Kobyzev A. Ghodsi 29 159 0 14 Oct 2022
Predicting Fine-Tuning Performance with Probing Zining Zhu Soroosh Shahtalebi Frank Rudzicz 30 9 0 13 Oct 2022
Language Model Decoding as Likelihood-Utility Alignment Martin Josifoski Maxime Peyrard Frano Rajic Jiheng Wei Debjit Paul ... Barun Patra Vishrav Chaudhary Emre Kıcıman Boi Faltings Robert West 40 4 0 13 Oct 2022