Towards a Unified View of Parameter-Efficient Transfer Learning

8 October 2021

Junxian He

Chunting Zhou

Xuezhe Ma

Taylor Berg-Kirkpatrick

Graham Neubig

AAML

ArXiv PDF HTML

Papers citing "Towards a Unified View of Parameter-Efficient Transfer Learning"

33 / 633 papers shown

Title
A Unified and Biologically-Plausible Relational Graph Representation of Vision Transformers Yuzhong Chen Yu Du Zhe Xiao Lin Zhao Lu Zhang ... Dajiang Zhu Tuo Zhang Xintao Hu Tianming Liu Xi Jiang ViT 27 5 0 20 May 2022
AdaVAE: Exploring Adaptive GPT-2s in Variational Auto-Encoders for Language Modeling Haoqin Tu Zhongliang Yang Jinshuai Yang Yong Huang 20 12 0 12 May 2022
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning Haokun Liu Derek Tam Mohammed Muqeeth Jay Mohta Tenghao Huang Joey Tianyi Zhou Colin Raffel 38 851 0 11 May 2022
Empowering parameter-efficient transfer learning by recognizing the kernel structure in self-attention Yifan Chen Devamanyu Hazarika Mahdi Namazifar Yang Liu Di Jin Dilek Z. Hakkani-Tür 24 9 0 07 May 2022
Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion Xiang Chen Ningyu Zhang Lei Li Shumin Deng Chuanqi Tan Changliang Xu Fei Huang Luo Si Huajun Chen 20 127 0 04 May 2022
Prompt Consistency for Zero-Shot Task Generalization Chunting Zhou Junxian He Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig VLM 23 74 0 29 Apr 2022
Parameter-efficient Model Adaptation for Vision Transformers Xuehai He Chunyuan Li Pengchuan Zhang Jianwei Yang Qing Guo 30 84 0 29 Mar 2022
Visual Prompt Tuning Menglin Jia Luming Tang Bor-Chun Chen Claire Cardie Serge Belongie Bharath Hariharan Ser-Nam Lim VLM VPVLM 40 1,533 0 23 Mar 2022
Continuous Detection, Rapidly React: Unseen Rumors Detection based on Continual Prompt-Tuning Yuhui Zuo Wei Zhu Guoyong Cai CLL VLM 26 11 0 16 Mar 2022
Hyperdecoders: Instance-specific decoders for multi-task NLP Hamish Ivison Matthew E. Peters AI4CE 28 20 0 15 Mar 2022
Modular and Parameter-Efficient Multimodal Fusion with Prompting Sheng Liang Mengjie Zhao Hinrich Schütze 38 42 0 15 Mar 2022
Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models Ning Ding Yujia Qin Guang Yang Fu Wei Zonghan Yang ... Jianfei Chen Yang Liu Jie Tang Juan Li Maosong Sun 32 196 0 14 Mar 2022
HyperPELT: Unified Parameter-Efficient Language Model Tuning for Both Language and Vision-and-Language Tasks Zhengkun Zhang Wenya Guo Xiaojun Meng Yasheng Wang Yadao Wang Xin Jiang Qun Liu Zhenglu Yang 34 15 0 08 Mar 2022
HyperPrompt: Prompt-based Task-Conditioning of Transformers Yun He H. Zheng Yi Tay Jai Gupta Yu Du ... Yaguang Li Zhaoji Chen Donald Metzler Heng-Tze Cheng Ed H. Chi LRM VLM 26 84 0 01 Mar 2022
Combining Modular Skills in Multitask Learning E. Ponti Alessandro Sordoni Yoshua Bengio Siva Reddy MoE 12 37 0 28 Feb 2022
Revisiting Parameter-Efficient Tuning: Are We Really There Yet? Guanzheng Chen Fangyu Liu Zaiqiao Meng Shangsong Liang 26 88 0 16 Feb 2022
ScaLA: Accelerating Adaptation of Pre-Trained Transformer-Based Language Models via Efficient Large-Batch Adversarial Noise Minjia Zhang U. Niranjan Yuxiong He 28 1 0 29 Jan 2022
Pretrained Language Models for Text Generation: A Survey Junyi Li Tianyi Tang Wayne Xin Zhao J. Nie Ji-Rong Wen AI4CE 36 127 0 14 Jan 2022
Black-Box Tuning for Language-Model-as-a-Service Tianxiang Sun Yunfan Shao Hong Qian Xuanjing Huang Xipeng Qiu VLM 50 256 0 10 Jan 2022
Efficient Hierarchical Domain Adaptation for Pretrained Language Models Alexandra Chronopoulou Matthew E. Peters Jesse Dodge 33 42 0 16 Dec 2021
VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks Yi-Lin Sung Jaemin Cho Joey Tianyi Zhou VLM VPVLM 35 343 0 13 Dec 2021
MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuning C. Eichenberg Sid Black Samuel Weinbach Letitia Parcalabescu Anette Frank MLLM VLM 15 100 0 09 Dec 2021
OpenPrompt: An Open-source Framework for Prompt-learning Ning Ding Shengding Hu Weilin Zhao Yulin Chen Zhiyuan Liu Haitao Zheng Maosong Sun VLM LLMAG 37 284 0 03 Nov 2021
DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models Xuxi Chen Tianlong Chen Weizhu Chen Ahmed Hassan Awadallah Zhangyang Wang Yu Cheng MoE ALM 20 10 0 30 Oct 2021
CLIP-Adapter: Better Vision-Language Models with Feature Adapters Peng Gao Shijie Geng Renrui Zhang Teli Ma Rongyao Fang Yongfeng Zhang Hongsheng Li Yu Qiao VLM CLIP 101 984 0 09 Oct 2021
PPT: Pre-trained Prompt Tuning for Few-shot Learning Yuxian Gu Xu Han Zhiyuan Liu Minlie Huang VLM 54 403 0 09 Sep 2021
Do Prompt-Based Models Really Understand the Meaning of their Prompts? Albert Webson Ellie Pavlick LRM 53 355 0 02 Sep 2021
YANMTT: Yet Another Neural Machine Translation Toolkit Raj Dabre Eiichiro Sumita 31 13 0 25 Aug 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,858 0 18 Apr 2021
KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction Xiang Chen Ningyu Zhang Xin Xie Shumin Deng Yunzhi Yao Chuanqi Tan Fei Huang Luo Si Huajun Chen 38 402 0 15 Apr 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,824 0 14 Dec 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 243 1,452 0 18 Mar 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 299 6,984 0 20 Apr 2018