Towards a Unified View of Parameter-Efficient Transfer Learning

8 October 2021

Junxian He

Chunting Zhou

Xuezhe Ma

Taylor Berg-Kirkpatrick

Graham Neubig

AAML

ArXiv PDF HTML

Papers citing "Towards a Unified View of Parameter-Efficient Transfer Learning"

50 / 633 papers shown

Title
Feature-Proxy Transformer for Few-Shot Segmentation Jianwei Zhang Yifan Sun Yi Yang Wei Chen ViT 27 61 0 13 Oct 2022
Continual Training of Language Models for Few-Shot Learning Zixuan Ke Haowei Lin Yijia Shao Hu Xu Lei Shu Bin Liu KELM BDL CLL 90 34 0 11 Oct 2022
Exploring Efficient-tuning Methods in Self-supervised Speech Models Zih-Ching Chen Chin-Lun Fu Chih-Ying Liu Shang-Wen Li Hung-yi Lee 24 38 0 10 Oct 2022
SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters Shwai He Liang Ding Daize Dong Miao Zhang Dacheng Tao MoE 29 87 0 09 Oct 2022
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models S. Kwon Jeonghoon Kim Jeongin Bae Kang Min Yoo Jin-Hwa Kim Baeseong Park Byeongwook Kim Jung-Woo Ha Nako Sung Dongsoo Lee MQ 29 30 0 08 Oct 2022
PCAE: A Framework of Plug-in Conditional Auto-Encoder for Controllable Text Generation Haoqin Tu Zhongliang Yang Jinshuai Yang Siyu Zhang Yong Huang 25 7 0 07 Oct 2022
Polyhistor: Parameter-Efficient Multi-Task Adaptation for Dense Vision Tasks Yen-Cheng Liu Chih-Yao Ma Junjiao Tian Zijian He Z. Kira 126 47 0 07 Oct 2022
Unsupervised Neural Stylistic Text Generation using Transfer learning and Adapters Vinayshekhar Bannihatti Kumar Rashmi Gangadharaiah Dan Roth 30 1 0 07 Oct 2022
Damage Control During Domain Adaptation for Transducer Based Automatic Speech Recognition Somshubra Majumdar Shantanu Acharya Vitaly Lavrukhin Boris Ginsburg 24 3 0 06 Oct 2022
Improving the Sample Efficiency of Prompt Tuning with Domain Adaptation Xu Guo Boyang Albert Li Han Yu VLM 39 22 0 06 Oct 2022
LPT: Long-tailed Prompt Tuning for Image Classification Bowen Dong Pan Zhou Shuicheng Yan W. Zuo VPVLM VLM 62 54 0 03 Oct 2022
Visual Prompt Tuning for Generative Transfer Learning Kihyuk Sohn Yuan Hao José Lezama Luisa F. Polanía Huiwen Chang Han Zhang Irfan Essa Lu Jiang VPVLM VLM 56 81 0 03 Oct 2022
Towards a Unified View on Visual Parameter-Efficient Transfer Learning Bruce X. B. Yu Jianlong Chang Lin Liu Qi Tian Changan Chen VPVLM VLM 68 34 0 03 Oct 2022
Differentially Private Bias-Term Fine-tuning of Foundation Models Zhiqi Bu Yu-Xiang Wang Sheng Zha George Karypis 28 46 0 30 Sep 2022
Towards Parameter-Efficient Integration of Pre-Trained Language Models In Temporal Video Grounding Erica K. Shimomoto Edison Marrese-Taylor Hiroya Takamura Ichiro Kobayashi Hideki Nakayama Yusuke Miyao 27 7 0 26 Sep 2022
EffEval: A Comprehensive Evaluation of Efficiency for MT Evaluation Metrics Daniil Larionov Jens Grunwald Christoph Leiter Steffen Eger 25 5 0 20 Sep 2022
Top-Tuning: a study on transfer learning for an efficient alternative to fine tuning for image classification with fast kernel methods P. D. Alfano Vito Paolo Pastore Lorenzo Rosasco Francesca Odone 21 6 0 16 Sep 2022
StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuation A. Maharana Darryl Hannan Joey Tianyi Zhou DiffM 34 77 0 13 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 30 109 0 31 Aug 2022
To Adapt or to Fine-tune: A Case Study on Abstractive Summarization Zheng Zhao Pinzhen Chen 19 3 0 30 Aug 2022
Few-Shot Table-to-Text Generation with Prefix-Controlled Generator Yutao Luo Menghua Lu Gongshen Liu Shilin Wang LMTD 21 8 0 23 Aug 2022
PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao VLM CLL 32 41 0 22 Aug 2022
Prompt-Matched Semantic Segmentation Lingbo Liu Jianlong Chang Bruce X. B. Yu Liang Lin Qi Tian Changrui Chen VPVLM VLM 25 27 0 22 Aug 2022
Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNets Hao Chen R. Tao Han Zhang Yidong Wang Xiang Li Weirong Ye Jindong Wang Guosheng Hu Marios Savvides VPVLM 29 52 0 15 Aug 2022
Disentangled Modeling of Domain and Relevance for Adaptable Dense Retrieval Jingtao Zhan Qingyao Ai Yiqun Liu Jiaxin Mao Xiaohui Xie Mengdi Zhang Shaoping Ma 30 10 0 11 Aug 2022
Improving Task Generalization via Unified Schema Prompt Wanjun Zhong Yifan Gao Ning Ding Zhiyuan Liu Ming Zhou Jiahai Wang Jian Yin Nan Duan 30 8 0 05 Aug 2022
Prompt Tuning for Generative Multimodal Pretrained Models Han Yang Junyang Lin An Yang Peng Wang Chang Zhou Hongxia Yang VLM LRM VPVLM 37 30 0 04 Aug 2022
Efficient Fine-Tuning of Compressed Language Models with Learners Danilo Vucetic Mohammadreza Tayaranian M. Ziaeefard J. Clark B. Meyer W. Gross 18 2 0 03 Aug 2022
Convolutional Bypasses Are Better Vision Transformer Adapters Shibo Jie Zhi-Hong Deng VPVLM 21 131 0 14 Jul 2022
Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation Zejiang Hou Julian Salazar George Polovets 30 14 0 07 Jul 2022
Probing via Prompting Jiaoda Li Ryan Cotterell Mrinmaya Sachan 37 13 0 04 Jul 2022
ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning Junting Pan Ziyi Lin Xiatian Zhu Jing Shao Hongsheng Li 27 190 0 27 Jun 2022
Sparse Structure Search for Parameter-Efficient Tuning Shengding Hu Zhen Zhang Ning Ding Yadao Wang Yasheng Wang Zhiyuan Liu Maosong Sun 34 16 0 15 Jun 2022
LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning Yi-Lin Sung Jaemin Cho Joey Tianyi Zhou VLM 21 236 0 13 Jun 2022
Neural Prompt Search Yuanhan Zhang Kaiyang Zhou Ziwei Liu VPVLM VLM 41 144 0 09 Jun 2022
Making Large Language Models Better Reasoners with Step-Aware Verifier Yifei Li Zeqi Lin Shizhuo Zhang Qiang Fu B. Chen Jian-Guang Lou Weizhu Chen ReLM LRM 38 210 0 06 Jun 2022
Parameter-Efficient and Student-Friendly Knowledge Distillation Jun Rao Xv Meng Liang Ding Shuhan Qi Dacheng Tao 37 46 0 28 May 2022
AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition Shoufa Chen Chongjian Ge Zhan Tong Jiangliu Wang Yibing Song Jue Wang Ping Luo 152 638 0 26 May 2022
Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation Tu Vu Aditya Barua Brian Lester Daniel Cer Mohit Iyyer Noah Constant CLL 23 64 0 25 May 2022
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning Mingkai Deng Jianyu Wang Cheng-Ping Hsieh Yihan Wang Han Guo Tianmin Shu Meng Song Eric P. Xing Zhiting Hu 27 319 0 25 May 2022
Memorization in NLP Fine-tuning Methods Fatemehsadat Mireshghallah Archit Uniyal Tianhao Wang David E. Evans Taylor Berg-Kirkpatrick AAML 61 39 0 25 May 2022
Know Where You're Going: Meta-Learning for Parameter-Efficient Fine-Tuning Mozhdeh Gheini Xuezhe Ma Jonathan May 42 5 0 25 May 2022
Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing Linlu Qiu Peter Shaw Panupong Pasupat Tianze Shi Jonathan Herzig Emily Pitler Fei Sha Kristina Toutanova AI4CE LRM 33 52 0 24 May 2022
Enhancing Continual Learning with Global Prototypes: Counteracting Negative Representation Drift Xueying Bai Jinghuan Shang Yifan Sun Niranjan Balasubramanian CLL 35 1 0 24 May 2022
Hyper-X: A Unified Hypernetwork for Multi-Task Multilingual Transfer Ahmet Üstün Arianna Bisazza G. Bouma Gertjan van Noord Sebastian Ruder 51 32 0 24 May 2022
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 130 100 0 24 May 2022
When does Parameter-Efficient Transfer Learning Work for Machine Translation? Ahmet Üstün Asa Cooper Stickland 37 7 0 23 May 2022
BBTv2: Towards a Gradient-Free Future with Large Language Models Tianxiang Sun Zhengfu He Hong Qian Yunhua Zhou Xuanjing Huang Xipeng Qiu 108 53 0 23 May 2022
Parameter-Efficient Sparsity for Large Language Models Fine-Tuning Yuchao Li Fuli Luo Chuanqi Tan Mengdi Wang Songfang Huang Shen Li Junjie Bai MQ 55 1 0 23 May 2022
FedAdapter: Efficient Federated Learning for Modern NLP Dongqi Cai Yaozong Wu Shangguang Wang F. Lin Mengwei Xu FedML AI4CE 36 21 0 20 May 2022