Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 965 papers shown

Title
Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation Mozhdeh Gheini Xiang Ren Jonathan May LRM 33 105 0 18 Apr 2021
What to Pre-Train on? Efficient Intermediate Task Selection Clifton A. Poth Jonas Pfeiffer Andreas Rucklé Iryna Gurevych 24 95 0 16 Apr 2021
MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning Mengzhou Xia Guoqing Zheng Subhabrata Mukherjee Milad Shokouhi Graham Neubig Ahmed Hassan Awadallah 33 31 0 16 Apr 2021
Relational World Knowledge Representation in Contextual Language Models: A Review Tara Safavi Danai Koutra KELM 43 51 0 12 Apr 2021
Continual Learning for Text Classification with Information Disentanglement Based Regularization Yufan Huang Yanzhe Zhang Jiaao Chen Xuezhi Wang Diyi Yang CLL 31 106 0 12 Apr 2021
Rainbow Memory: Continual Learning with a Memory of Diverse Samples Jihwan Bang Heesu Kim Y. Yoo Jung-Woo Ha Jonghyun Choi CLL 42 324 0 31 Mar 2021
BASE Layers: Simplifying Training of Large, Sparse Models M. Lewis Shruti Bhosale Tim Dettmers Naman Goyal Luke Zettlemoyer MoE 53 274 0 30 Mar 2021
Attribute Alignment: Controlling Text Generation from Pre-trained Language Models Dian Yu Zhou Yu Kenji Sagae 21 37 0 20 Mar 2021
Structural Adapters in Pretrained Language Models for AMR-to-text Generation Leonardo F. R. Ribeiro Yue Zhang Iryna Gurevych 41 69 0 16 Mar 2021
Pretrained Transformers as Universal Computation Engines Kevin Lu Aditya Grover Pieter Abbeel Igor Mordatch 30 218 0 09 Mar 2021
NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Task Muhammad Abdul-Mageed Chiyu Zhang AbdelRahim Elmadany Houda Bouamor Nizar Habash 23 75 0 04 Mar 2021
Random Feature Attention Hao Peng Nikolaos Pappas Dani Yogatama Roy Schwartz Noah A. Smith Lingpeng Kong 38 350 0 03 Mar 2021
Adapting MARBERT for Improved Arabic Dialect Identification: Submission to the NADI 2021 Shared Task Badr AlKhamissi Mohamed Gabr Muhammad N. ElNokrashy Khaled Essam 18 17 0 01 Mar 2021
Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezing Charlie Blake Vitaly Kurin Maximilian Igl Shimon Whiteson AI4CE 23 13 0 01 Mar 2021
Self-Tuning for Data-Efficient Deep Learning Ximei Wang Jing Gao Mingsheng Long Jianmin Wang BDL 30 70 0 25 Feb 2021
Meta-Transfer Learning for Low-Resource Abstractive Summarization Yi-Syuan Chen Hong-Han Shuai CLL OffRL 48 38 0 18 Feb 2021
AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuning Yuhan Liu Saurabh Agarwal Shivaram Venkataraman OffRL 22 54 0 02 Feb 2021
Combining pre-trained language models and structured knowledge Pedro Colon-Hernandez Catherine Havasi Jason B. Alonso Matthew Huggins C. Breazeal KELM 43 48 0 28 Jan 2021
Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing Minh Nguyen Viet Dac Lai Amir Pouran Ben Veyseh Thien Huu Nguyen 52 132 0 09 Jan 2021
Learning to Generate Task-Specific Adapters from Task Description Qinyuan Ye Xiang Ren 117 29 0 02 Jan 2021
Analyzing Commonsense Emergence in Few-shot Knowledge Models Jeff Da Ronan Le Bras Ximing Lu Yejin Choi Antoine Bosselut AI4MH KELM 77 40 0 01 Jan 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 61 4,103 0 01 Jan 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 254 342 0 01 Jan 2021
UNKs Everywhere: Adapting Multilingual Language Models to New Scripts Jonas Pfeiffer Ivan Vulić Iryna Gurevych Sebastian Ruder 29 128 0 31 Dec 2020
Continual Learning in Task-Oriented Dialogue Systems Andrea Madotto Zhaojiang Lin Zhenpeng Zhou Seungwhan Moon Paul A. Crook Bing-Quan Liu Zhou Yu Eunjoon Cho Zhiguang Wang CLL 43 127 0 31 Dec 2020
Few-shot Sequence Learning with Transformers Lajanugen Logeswaran Ann Lee Myle Ott Honglak Lee MarcÁurelio Ranzato Arthur Szlam ViT 46 12 0 17 Dec 2020
Parameter-Efficient Transfer Learning with Diff Pruning Demi Guo Alexander M. Rush Yoon Kim 17 386 0 14 Dec 2020
Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer M. Vidoni Ivan Vulić Goran Glavaš 33 27 0 11 Dec 2020
Efficient Estimation of Influence of a Training Instance Sosuke Kobayashi Sho Yokoi Jun Suzuki Kentaro Inui TDI 37 15 0 08 Dec 2020
Modifying Memories in Transformer Models Chen Zhu A. S. Rawat Manzil Zaheer Srinadh Bhojanapalli Daliang Li Felix X. Yu Sanjiv Kumar KELM 32 192 0 01 Dec 2020
Emergent Communication Pretraining for Few-Shot Machine Translation Yaoyiran Li Edoardo Ponti Ivan Vulić Anna Korhonen 27 19 0 02 Nov 2020
Target Word Masking for Location Metonymy Resolution Haonan Li Maria Vasardani Martin Tomko Timothy Baldwin 22 11 0 30 Oct 2020
The LMU Munich System for the WMT 2020 Unsupervised Machine Translation Shared Task Alexandra Chronopoulou Dario Stojanovski Viktor Hangya Alexander Fraser 37 5 0 25 Oct 2020
AdapterDrop: On the Efficiency of Adapters in Transformers Andreas Rucklé Gregor Geigle Max Glockner Tilman Beck Jonas Pfeiffer Nils Reimers Iryna Gurevych 57 255 0 22 Oct 2020
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 244 612 0 13 Oct 2020
Style Attuned Pre-training and Parameter Efficient Fine-tuning for Spoken Language Understanding Jin Cao Jun Wang Wael Hamza Kelly Vanee Shang-Wen Li 19 10 0 09 Oct 2020
MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale Andreas Rucklé Jonas Pfeiffer Iryna Gurevych 27 37 0 02 Oct 2020
Scalable Transfer Learning with Expert Models J. Puigcerver C. Riquelme Basil Mustafa Cédric Renggli André Susano Pinto Sylvain Gelly Daniel Keysers N. Houlsby 34 63 0 28 Sep 2020
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data Jonathan Pilault Amine Elhattami C. Pal CLL MoE 30 89 0 19 Sep 2020
Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT Alexandra Chronopoulou Dario Stojanovski Alexander Fraser 21 33 0 16 Sep 2020
Efficient Transformers: A Survey Yi Tay Mostafa Dehghani Dara Bahri Donald Metzler VLM 116 1,104 0 14 Sep 2020
Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature Jongjin Park CLL 33 15 0 07 Jul 2020
Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization Sang Michael Xie Tengyu Ma Percy Liang 40 13 0 29 Jun 2020
Revisiting Few-sample BERT Fine-tuning Tianyi Zhang Felix Wu Arzoo Katiyar Kilian Q. Weinberger Yoav Artzi 41 442 0 10 Jun 2020
A Survey on Transfer Learning in Natural Language Processing Zaid Alyafeai Maged S. Alshaibani Irfan Ahmad 30 72 0 31 May 2020
Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers Anne Lauscher Olga Majewska Leonardo F. R. Ribeiro Iryna Gurevych Nikolai Rozanov Goran Glavaš KELM 39 79 0 24 May 2020
AdapterFusion: Non-Destructive Task Composition for Transfer Learning Jonas Pfeiffer Aishwarya Kamath Andreas Rucklé Kyunghyun Cho Iryna Gurevych CLL MoMe 56 820 0 01 May 2020
Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation Asa Cooper Stickland Xian Li Marjan Ghazvininejad 36 44 0 30 Apr 2020
UDapter: Language Adaptation for Truly Universal Dependency Parsing Ahmet Üstün Arianna Bisazza G. Bouma Gertjan van Noord 29 113 0 29 Apr 2020
Fine-tuning Multi-hop Question Answering with Hierarchical Graph Network Guanming Xiong 26 0 0 20 Apr 2020