v1v2 (latest)

Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 2,860 papers shown

Title
Example-based Hypernetworks for Out-of-Distribution Generalization Tomer Volk Eyal Ben-David Ohad Amosy Gal Chechik Roi Reichart OOD 93 20 0 27 Mar 2022
FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulations Mirian Hipolito Garcia Andre Manoel Daniel Madrigal Diaz Fatemehsadat Mireshghallah Robert Sim Dimitrios Dimitriadis FedML 92 57 0 25 Mar 2022
UKP-SQUARE: An Online Platform for Question Answering Research Tim Baumgärtner Kexin Wang Rachneet Sachdeva Max Eichler Gregor Geigle ... Leonardo F. R. Ribeiro Jonas Pfeiffer Nils Reimers Gözde Gül Sahin Iryna Gurevych 64 7 0 25 Mar 2022
Shoring Up the Foundations: Fusing Model Embeddings and Weak Supervision Mayee F. Chen Daniel Y. Fu Dyah Adila Michael Zhang Frederic Sala Kayvon Fatahalian Christopher Ré 78 20 0 24 Mar 2022
A Scalable Model Specialization Framework for Training and Inference using Submodels and its Application to Speech Model Personalization Fadi Biadsy Youzheng Chen Xia Zhang Oleg Rybakov Andrew Rosenberg Pedro J. Moreno 122 13 0 23 Mar 2022
Pathways: Asynchronous Distributed Dataflow for ML P. Barham Aakanksha Chowdhery J. Dean Sanjay Ghemawat Steven Hand ... Parker Schuh Ryan Sepassi Laurent El Shafey C. A. Thekkath Yonghui Wu GNN MoE 125 133 0 23 Mar 2022
Adaptive Transformers for Robust Few-shot Cross-domain Face Anti-spoofing Hsin-Ping Huang Deqing Sun Yaojie Liu Wen-Sheng Chu Taihong Xiao Jinwei Yuan Hartwig Adam Ming-Hsuan Yang CVBM 117 62 0 23 Mar 2022
Visual Prompt Tuning Menglin Jia Luming Tang Bor-Chun Chen Claire Cardie Serge Belongie Bharath Hariharan Ser-Nam Lim VLM VPVLM 262 1,661 0 23 Mar 2022
Meta-attention for ViT-backed Continual Learning Mengqi Xue Haofei Zhang Mingli Song Mingli Song CLL 80 43 0 22 Mar 2022
Continual Sequence Generation with Adaptive Compositional Modules Yanzhe Zhang Xuezhi Wang Diyi Yang KELM CLL 102 43 0 20 Mar 2022
Hierarchical Inductive Transfer for Continual Dialogue Learning Shaoxiong Feng Xuancheng Ren Kan Li Xu Sun CLL 63 4 0 20 Mar 2022
On Robust Prefix-Tuning for Text Classification Zonghan Yang Yang Liu VLM 78 21 0 19 Mar 2022
$Meta-X$_{NLG}$: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation$ Meta-X $_{NLG}$ : A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation Kaushal Kumar Maurya M. Desarkar 85 8 0 19 Mar 2022
Three things everyone should know about Vision Transformers Hugo Touvron Matthieu Cord Alaaeldin El-Nouby Jakob Verbeek Hervé Jégou ViT 121 123 0 18 Mar 2022
Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer Huiyuan Lai Antonio Toral Malvina Nissim 61 13 0 16 Mar 2022
Continuous Detection, Rapidly React: Unseen Rumors Detection based on Continual Prompt-Tuning Yuhui Zuo Wei Zhu Guoyong Cai CLL VLM 90 11 0 16 Mar 2022
Hyperdecoders: Instance-specific decoders for multi-task NLP Hamish Ivison Matthew E. Peters AI4CE 121 22 0 15 Mar 2022
Modular and Parameter-Efficient Multimodal Fusion with Prompting Sheng Liang Mengjie Zhao Hinrich Schütze 98 45 0 15 Mar 2022
Graph Pre-training for AMR Parsing and Generation Xuefeng Bai Yulong Chen Yue Zhang SSL 119 103 0 15 Mar 2022
GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models Archiki Prasad Peter Hase Xiang Zhou Joey Tianyi Zhou 133 124 0 14 Mar 2022
Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models Ning Ding Yujia Qin Guang Yang Fu Wei Zonghan Yang ... Jianfei Chen Yang Liu Jie Tang Juan Li Maosong Sun 136 205 0 14 Mar 2022
Towards Personalized Intelligence at Scale Yiping Kang Ashish Mahendra Christopher Clarke Lingjia Tang Jason Mars 81 1 0 13 Mar 2022
Continual Prompt Tuning for Dialog State Tracking Qi Zhu Bing Li Fei Mi Xiaoyan Zhu Minlie Huang CLL KELM 94 60 0 13 Mar 2022
Memory Efficient Continual Learning with Transformers Beyza Ermis Giovanni Zappella Martin Wistuba Aditya Rawal Cédric Archambeau CLL 84 46 0 09 Mar 2022
Adaptor: Objective-Centric Adaptation Framework for Language Models Michal vStefánik Vít Novotný Nikola Groverová Petr Sojka 99 10 0 08 Mar 2022
HyperPELT: Unified Parameter-Efficient Language Model Tuning for Both Language and Vision-and-Language Tasks Zhengkun Zhang Wenya Guo Xiaojun Meng Yasheng Wang Yadao Wang Xin Jiang Qun Liu Zhenglu Yang 80 17 0 08 Mar 2022
HyperMixer: An MLP-based Low Cost Alternative to Transformers Florian Mai Arnaud Pannatier Fabio Fehr Haolin Chen François Marelli François Fleuret James Henderson 97 12 0 07 Mar 2022
Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models Shengnan An Yifei Li Zeqi Lin Qian Liu Bei Chen Qiang Fu Weizhu Chen Nanning Zheng Jian-Guang Lou VLM AAML 97 43 0 07 Mar 2022
Unfreeze with Care: Space-Efficient Fine-Tuning of Semantic Parsing Models Weiqi Sun Haidar Khan Nicolas Guenon des Mesnards M. Rubino Konstantine Arkoudas 124 5 0 05 Mar 2022
Controlling the Focus of Pretrained Language Generation Models Jiabao Ji Yoon Kim James R. Glass Tianxing He 121 5 0 02 Mar 2022
Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Models Ze-Feng Gao Peiyu Liu Wayne Xin Zhao Zhong-Yi Lu Ji-Rong Wen MoE 73 27 0 02 Mar 2022
HyperPrompt: Prompt-based Task-Conditioning of Transformers Yun He H. Zheng Yi Tay Jai Gupta Yu Du ... Yaguang Li Zhaoji Chen Donald Metzler Heng-Tze Cheng Ed H. Chi LRM VLM 113 93 0 01 Mar 2022
Combining Modular Skills in Multitask Learning Edoardo Ponti Alessandro Sordoni Yoshua Bengio Siva Reddy MoE 89 38 0 28 Feb 2022
Cross-Lingual Text Classification with Multilingual Distillation and Zero-Shot-Aware Training Ziqing Yang Yiming Cui Zhigang Chen Shijin Wang VLM 141 3 0 28 Feb 2022
BERTVision -- A Parameter-Efficient Approach for Question Answering Siduo Jiang Cristopher Benge Will King 41 1 0 24 Feb 2022
BERT WEAVER: Using WEight AVERaging to enable lifelong learning for transformer-based models in biomedical semantic search engines Lisa Langnickel Alexander Schulz Barbara Hammer Juliane Fluck CLL MedIm 79 3 0 21 Feb 2022
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution Ananya Kumar Aditi Raghunathan Robbie Jones Tengyu Ma Percy Liang OODD 177 690 0 21 Feb 2022
$$\mathcal{Y}$-Tuning: An Efficient Tuning Paradigm for Large-Scale Pre-Trained Models via Label Representation Learning$ $\mathcal{Y}$ -Tuning: An Efficient Tuning Paradigm for Large-Scale Pre-Trained Models via Label Representation Learning Yitao Liu Chen An Xipeng Qiu 93 18 0 20 Feb 2022
TURNER: The Uncertainty-based Retrieval Framework for Chinese NER Zhichao Geng Hang Yan Zhangyue Yin Chen An Xipeng Qiu UQLM 66 6 0 18 Feb 2022
ST-MoE: Designing Stable and Transferable Sparse Expert Models Barret Zoph Irwan Bello Sameer Kumar Nan Du Yanping Huang J. Dean Noam M. Shazeer W. Fedus MoE 229 205 0 17 Feb 2022
FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction Minh Le Nguyen Nghia Trung Ngo Bonan Min Thien Huu Nguyen 78 11 0 16 Feb 2022
Revisiting Parameter-Efficient Tuning: Are We Really There Yet? Guanzheng Chen Fangyu Liu Zaiqiao Meng Shangsong Liang 68 95 0 16 Feb 2022
EdgeFormer: A Parameter-Efficient Transformer for On-Device Seq2seq Generation Tao Ge Si-Qing Chen Furu Wei MoE 97 23 0 16 Feb 2022
I-Tuning: Tuning Frozen Language Models with Image for Lightweight Image Captioning Ziyang Luo Zhipeng Hu Yadong Xi Rongsheng Zhang Jing Ma VLM 57 14 0 14 Feb 2022
Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding Peter Sullivan Toshiko Shibano Muhammad Abdul-Mageed 83 11 0 10 Feb 2022
Can Open Domain Question Answering Systems Answer Visual Knowledge Questions? Jiawen Zhang Abhijit Mishra Avinesh P.V.S Siddharth Patwardhan Sachin Agarwal 77 0 0 09 Feb 2022
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models Wei Ping Ming-Yu Liu Chaowei Xiao Peng Xu M. Patwary Mohammad Shoeybi Yue Liu Anima Anandkumar Bryan Catanzaro 104 71 0 08 Feb 2022
Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognition Bethan Thomas Samuel Kessler S. Karout 73 72 0 07 Feb 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 268 884 0 07 Feb 2022
Towards Coherent and Consistent Use of Entities in Narrative Generation Pinelopi Papalampidi Kris Cao Tomás Kociský HILM 61 13 0 03 Feb 2022