v1v2 (latest)

Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 2,860 papers shown

Title
Continual Learning for Text Classification with Information Disentanglement Based Regularization Yufan Huang Yanzhe Zhang Jiaao Chen Xuezhi Wang Diyi Yang CLL 77 113 0 12 Apr 2021
Rainbow Memory: Continual Learning with a Memory of Diverse Samples Jihwan Bang Heesu Kim Y. Yoo Jung-Woo Ha Jonghyun Choi CLL 141 347 0 31 Mar 2021
BASE Layers: Simplifying Training of Large, Sparse Models M. Lewis Shruti Bhosale Tim Dettmers Naman Goyal Luke Zettlemoyer MoE 224 285 0 30 Mar 2021
Attribute Alignment: Controlling Text Generation from Pre-trained Language Models Dian Yu Zhou Yu Kenji Sagae 82 40 0 20 Mar 2021
Structural Adapters in Pretrained Language Models for AMR-to-text Generation Leonardo F. R. Ribeiro Yue Zhang Iryna Gurevych 100 72 0 16 Mar 2021
Pretrained Transformers as Universal Computation Engines Kevin Lu Aditya Grover Pieter Abbeel Igor Mordatch 92 221 0 09 Mar 2021
NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Task Muhammad Abdul-Mageed Chiyu Zhang AbdelRahim Elmadany Houda Bouamor Nizar Habash 105 78 0 04 Mar 2021
Random Feature Attention Hao Peng Nikolaos Pappas Dani Yogatama Roy Schwartz Noah A. Smith Lingpeng Kong 148 362 0 03 Mar 2021
Adapting MARBERT for Improved Arabic Dialect Identification: Submission to the NADI 2021 Shared Task Badr AlKhamissi Mohamed Gabr Muhammad N. ElNokrashy Khaled Essam 89 20 0 01 Mar 2021
Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezing Charlie Blake Vitaly Kurin Maximilian Igl Shimon Whiteson AI4CE 100 13 0 01 Mar 2021
Self-Tuning for Data-Efficient Deep Learning Ximei Wang Jing Gao Mingsheng Long Jianmin Wang BDL 91 71 0 25 Feb 2021
Meta-Transfer Learning for Low-Resource Abstractive Summarization Yi-Syuan Chen Hong-Han Shuai CLL OffRL 103 39 0 18 Feb 2021
Improved Customer Transaction Classification using Semi-Supervised Knowledge Distillation Rohan Sukumaran 31 2 0 15 Feb 2021
A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages Jivnesh Sandhan Amrith Krishna Ashim Gupta Laxmidhar Behera Pawan Goyal 51 9 0 12 Feb 2021
CSS-LM: A Contrastive Framework for Semi-supervised Fine-tuning of Pre-trained Language Models Yusheng Su Xu Han Yankai Lin Zhengyan Zhang Zhiyuan Liu Peng Li Jie Zhou Maosong Sun 73 10 0 07 Feb 2021
AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuning Yuhan Liu Saurabh Agarwal Shivaram Venkataraman OffRL 80 56 0 02 Feb 2021
Combining pre-trained language models and structured knowledge Pedro Colon-Hernandez Catherine Havasi Jason B. Alonso Matthew Huggins C. Breazeal KELM 93 48 0 28 Jan 2021
Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing Minh Nguyen Viet Dac Lai Amir Pouran Ben Veyseh Thien Huu Nguyen 141 137 0 09 Jan 2021
Learning to Generate Task-Specific Adapters from Task Description Qinyuan Ye Xiang Ren 200 32 0 02 Jan 2021
Analyzing Commonsense Emergence in Few-shot Knowledge Models Jeff Da Ronan Le Bras Ximing Lu Yejin Choi Antoine Bosselut AI4MH KELM 186 41 0 01 Jan 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 257 4,339 0 01 Jan 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 358 354 0 01 Jan 2021
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 169 256 0 31 Dec 2020
UNKs Everywhere: Adapting Multilingual Language Models to New Scripts Jonas Pfeiffer Ivan Vulić Iryna Gurevych Sebastian Ruder 117 134 0 31 Dec 2020
Continual Learning in Task-Oriented Dialogue Systems Andrea Madotto Zhaojiang Lin Zhenpeng Zhou Seungwhan Moon Paul A. Crook Bing-Quan Liu Zhou Yu Eunjoon Cho Zhiguang Wang CLL 143 132 0 31 Dec 2020
Verb Knowledge Injection for Multilingual Event Processing Olga Majewska Ivan Vulić Goran Glavaš Edoardo Ponti Anna Korhonen 85 11 0 31 Dec 2020
Red Dragon AI at TextGraphs 2020 Shared Task: LIT : LSTM-Interleaved Transformer for Multi-Hop Explanation Ranking Yew Ken Chia Sam Witteveen Martin Andrews 35 4 0 28 Dec 2020
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning Armen Aghajanyan Luke Zettlemoyer Sonal Gupta 110 579 1 22 Dec 2020
Continual Lifelong Learning in Natural Language Processing: A Survey Magdalena Biesialska Katarzyna Biesialska Marta R. Costa-jussá KELM CLL 114 222 0 17 Dec 2020
Few-shot Sequence Learning with Transformers Lajanugen Logeswaran Ann Lee Myle Ott Honglak Lee MarcÁurelio Ranzato Arthur Szlam ViT 69 12 0 17 Dec 2020
Parameter-Efficient Transfer Learning with Diff Pruning Demi Guo Alexander M. Rush Yoon Kim 95 407 0 14 Dec 2020
Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer M. Vidoni Ivan Vulić Goran Glavaš 105 27 0 11 Dec 2020
Efficient Estimation of Influence of a Training Instance Sosuke Kobayashi Sho Yokoi Jun Suzuki Kentaro Inui TDI 89 15 0 08 Dec 2020
Adaptive Deep Learning for Entity Resolution by Risk Analysis Zhaoqiang Chen Qun Chen Youcef Nafa Tianyi Duan Wei Pan Lijun Zhang Zhanhuai Li 126 8 0 07 Dec 2020
Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognition Genta Indra Winata Guangsen Wang Caiming Xiong Guosheng Lin VLM 65 50 0 03 Dec 2020
Modifying Memories in Transformer Models Chen Zhu A. S. Rawat Manzil Zaheer Srinadh Bhojanapalli Daliang Li Felix X. Yu Sanjiv Kumar KELM 133 203 0 01 Dec 2020
Emergent Communication Pretraining for Few-Shot Machine Translation Yaoyiran Li Edoardo Ponti Ivan Vulić Anna Korhonen 106 19 0 02 Nov 2020
Target Word Masking for Location Metonymy Resolution Haonan Li Maria Vasardani Martin Tomko Timothy Baldwin 29 11 0 30 Oct 2020
The LMU Munich System for the WMT 2020 Unsupervised Machine Translation Shared Task Alexandra Chronopoulou Dario Stojanovski Viktor Hangya Alexander Fraser 103 5 0 25 Oct 2020
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models Benjamin Muller Antonis Anastasopoulos Benoît Sagot Djamé Seddah LRM 241 170 0 24 Oct 2020
AdapterDrop: On the Efficiency of Adapters in Transformers Andreas Rucklé Gregor Geigle Max Glockner Tilman Beck Jonas Pfeiffer Nils Reimers Iryna Gurevych 131 267 0 22 Oct 2020
Beyond English-Centric Multilingual Machine Translation Angela Fan Shruti Bhosale Holger Schwenk Zhiyi Ma Ahmed El-Kishky ... Vitaliy Liptchinsky Sergey Edunov Edouard Grave Michael Auli Armand Joulin LRM 102 865 0 21 Oct 2020
Update Frequently, Update Fast: Retraining Semantic Parsing Systems in a Fraction of Time Vladislav Lialin Rahul Goel A. Simanovsky Anna Rumshisky Rushin Shah CLL 35 1 0 15 Oct 2020
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 419 628 0 13 Oct 2020
Incorporating BERT into Parallel Sequence Decoding with Adapters Junliang Guo Zhirui Zhang Linli Xu Hao-Ran Wei Boxing Chen Enhong Chen 118 69 0 13 Oct 2020
Style Attuned Pre-training and Parameter Efficient Fine-tuning for Spoken Language Understanding Jin Cao Jun Wang Wael Hamza Kelly Vanee Shang-Wen Li 30 10 0 09 Oct 2020
Plug-and-Play Conversational Models Andrea Madotto Etsuko Ishii Zhaojiang Lin Sumanth Dathathri Pascale Fung 93 51 0 09 Oct 2020
CATBERT: Context-Aware Tiny BERT for Detecting Social Engineering Emails Younghoon Lee Joshua Saxe Richard E. Harang 51 25 0 07 Oct 2020
On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment Zirui Wang Zachary Chase Lipton Yulia Tsvetkov 93 32 0 06 Oct 2020
Efficient Meta Lifelong-Learning with Limited Memory Zirui Wang Sanket Vaibhav Mehta Barnabás Póczós J. Carbonell CLL KELM 81 76 0 06 Oct 2020