Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 965 papers shown

Title
Eliciting and Understanding Cross-Task Skills with Task-Level Mixture-of-Experts Qinyuan Ye Juan Zha Xiang Ren MoE 23 12 0 25 May 2022
Memorization in NLP Fine-tuning Methods Fatemehsadat Mireshghallah Archit Uniyal Tianhao Wang David Evans Taylor Berg-Kirkpatrick AAML 70 39 0 25 May 2022
Know Where You're Going: Meta-Learning for Parameter-Efficient Fine-Tuning Mozhdeh Gheini Xuezhe Ma Jonathan May 65 5 0 25 May 2022
Enhancing Continual Learning with Global Prototypes: Counteracting Negative Representation Drift Xueying Bai Jinghuan Shang Yifan Sun Niranjan Balasubramanian CLL 35 1 0 24 May 2022
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 130 100 0 24 May 2022
When does Parameter-Efficient Transfer Learning Work for Machine Translation? Ahmet Üstün Asa Cooper Stickland 47 7 0 23 May 2022
BBTv2: Towards a Gradient-Free Future with Large Language Models Tianxiang Sun Zhengfu He Hong Qian Yunhua Zhou Xuanjing Huang Xipeng Qiu 108 53 0 23 May 2022
muNet: Evolving Pretrained Deep Neural Networks into Scalable Auto-tuning Multitask Systems Andrea Gesmundo J. Dean 41 19 0 22 May 2022
Self-Supervised Speech Representation Learning: A Review Abdel-rahman Mohamed Hung-yi Lee Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin ... Shang-Wen Li Karen Livescu Lars Maaløe Tara N. Sainath Shinji Watanabe SSL AI4TS 137 354 0 21 May 2022
Can Foundation Models Wrangle Your Data? A. Narayan Ines Chami Laurel J. Orr Simran Arora Christopher Ré LMTD AI4CE 181 214 0 20 May 2022
Phylogeny-Inspired Adaptation of Multilingual Models to New Languages Fahim Faisal Antonios Anastasopoulos AI4CE LRM 34 26 0 19 May 2022
Nebula-I: A General Framework for Collaboratively Training Deep Learning Models on Low-Bandwidth Cloud Clusters Yang Xiang Zhihua Wu Weibao Gong Siyu Ding Xianjie Mo ... Yue Yu Ge Li Yu Sun Yanjun Ma Dianhai Yu 29 5 0 19 May 2022
Classification of Astronomical Bodies by Efficient Layer Fine-Tuning of Deep Neural Networks Sabeesh Ethiraj B. Bolla 20 8 0 14 May 2022
Unified Modeling of Multi-Domain Multi-Device ASR Systems Soumyajit Mitra Swayambhu Nath Ray Bharat Padi Arunasish Sen Raghavendra Bilgi Harish Arsikere Shalini Ghosh A. Srinivasamurthy Sri Garimella 42 3 0 13 May 2022
Lifting the Curse of Multilinguality by Pre-training Modular Transformers Jonas Pfeiffer Naman Goyal Xi Lin Xian Li James Cross Sebastian Riedel Mikel Artetxe LRM 40 140 0 12 May 2022
Extracting Latent Steering Vectors from Pretrained Language Models Nishant Subramani Nivedita Suresh Matthew E. Peters LLMSV 36 82 0 10 May 2022
Automated Evaluation for Student Argumentative Writing: A Survey Xinyu Wang Yohan Lee Juneyoung Park 16 12 0 09 May 2022
Efficient Few-Shot Fine-Tuning for Opinion Summarization Arthur Bravzinskas Ramesh Nallapati Joey Tianyi Zhou Markus Dreyer 19 24 0 04 May 2022
Training Mixed-Domain Translation Models via Federated Learning Peyman Passban Tanya Roosta Rahul Gupta Ankit R. Chadha Clement Chung FedML AI4CE 34 18 0 03 May 2022
Adaptable Adapters N. Moosavi Quentin Delfosse Kristian Kersting Iryna Gurevych 56 21 0 03 May 2022
Embedding Hallucination for Few-Shot Language Fine-tuning Yiren Jian Chongyang Gao Soroush Vosoughi 33 4 0 03 May 2022
AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks Chin-Lun Fu Zih-Ching Chen Yun-Ru Lee Hung-yi Lee 38 44 0 30 Apr 2022
Prompt Consistency for Zero-Shot Task Generalization Chunting Zhou Junxian He Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig VLM 26 74 0 29 Apr 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 53 3,369 0 29 Apr 2022
Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation Kexin Yang Dayiheng Liu Wenqiang Lei Baosong Yang Mingfeng Xue Boxing Chen Jun Xie 38 29 0 28 Apr 2022
Super-Prompting: Utilizing Model-Independent Contextual Data to Reduce Data Annotation Required in Visual Commonsense Tasks Navid Rezaei Marek Reformat VLM 17 2 0 25 Apr 2022
Identifying Chinese Opinion Expressions with Extremely-Noisy Crowdsourcing Annotations Xin Zhang Guangwei Xu Yueheng Sun Meishan Zhang Xiaobin Wang Hao Fei 42 11 0 22 Apr 2022
KALA: Knowledge-Augmented Language Model Adaptation Minki Kang Jinheon Baek Sung Ju Hwang VLM KELM 36 34 0 22 Apr 2022
CodexDB: Generating Code for Processing SQL Queries using GPT-3 Codex Immanuel Trummer LMTD 29 19 0 19 Apr 2022
ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models Chunyuan Li Haotian Liu Liunian Harold Li Pengchuan Zhang J. Aneja ... Ping Jin Houdong Hu Zicheng Liu Yong Jae Lee Jianfeng Gao 50 145 0 19 Apr 2022
VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance Katherine Crowson Stella Biderman Daniel Kornis Dashiell Stander Eric Hallahan Louis Castricato Edward Raff CLIP 80 371 0 18 Apr 2022
Learning to Express in Knowledge-Grounded Conversation Xueliang Zhao Tingchen Fu Chongyang Tao Wei Wu Dongyan Zhao Rui Yan 33 6 0 12 Apr 2022
A Call for Clarity in Beam Search: How It Works and When It Stops Jungo Kasai Keisuke Sakaguchi Ronan Le Bras Dragomir R. Radev Yejin Choi Noah A. Smith 28 6 0 11 Apr 2022
IDPG: An Instance-Dependent Prompt Generation Method Zhuofeng Wu Sinong Wang Jiatao Gu Rui Hou Yuxiao Dong V. Vydiswaran Hao Ma VLM 40 58 0 09 Apr 2022
Fair and Argumentative Language Modeling for Computational Argumentation Carolin Holtermann Anne Lauscher Simone Paolo Ponzetto 29 21 0 08 Apr 2022
Federated Learning with Partial Model Personalization Krishna Pillutla Kshitiz Malik Abdel-rahman Mohamed Michael G. Rabbat Maziar Sanjabi Lin Xiao FedML 43 157 0 08 Apr 2022
Parameter-Efficient Abstractive Question Answering over Tables or Text Vaishali Pal Evangelos Kanoulas Maarten de Rijke LMTD 27 14 0 07 Apr 2022
Domain Adaptation for Time-Series Classification to Mitigate Covariate Shift Felix Ott David Rügamer Lucas Heublein Bernd Bischl Christopher Mutschler OOD TTA AI4TS 38 31 0 07 Apr 2022
Genre-conditioned Acoustic Models for Automatic Lyrics Transcription of Polyphonic Music Xiaoxue Gao Chitralekha Gupta Haizhou Li 32 21 0 07 Apr 2022
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval Robert Litschko Ivan Vulić Goran Glavaš LRM 47 14 0 05 Apr 2022
SHiFT: An Efficient, Flexible Search Engine for Transfer Learning Cédric Renggli Xiaozhe Yao Luka Kolar Luka Rimanic Ana Klimovic Ce Zhang OOD 43 4 0 04 Apr 2022
PERFECT: Prompt-free and Efficient Few-shot Learning with Language Models Rabeeh Karimi Mahabadi Luke Zettlemoyer James Henderson Marzieh Saeidi Lambert Mathias Ves Stoyanov Majid Yazdani VLM 34 70 0 03 Apr 2022
Proper Reuse of Image Classification Features Improves Object Detection C. N. Vasconcelos Vighnesh Birodkar Vincent Dumoulin VLM 25 32 0 01 Apr 2022
Parameter-efficient Model Adaptation for Vision Transformers Xuehai He Chunyuan Li Pengchuan Zhang Jianwei Yang Xinze Wang 35 85 0 29 Mar 2022
Fine-tuning Image Transformers using Learnable Memory Mark Sandler A. Zhmoginov Max Vladymyrov Andrew Jackson ViT 34 47 0 29 Mar 2022
A Scalable Model Specialization Framework for Training and Inference using Submodels and its Application to Speech Model Personalization Fadi Biadsy Youzheng Chen Xia Zhang Oleg Rybakov Andrew Rosenberg Pedro J. Moreno 51 13 0 23 Mar 2022
Pathways: Asynchronous Distributed Dataflow for ML P. Barham Aakanksha Chowdhery J. Dean Sanjay Ghemawat Steven Hand ... Parker Schuh Ryan Sepassi Laurent El Shafey C. A. Thekkath Yonghui Wu GNN MoE 47 126 0 23 Mar 2022
Meta-attention for ViT-backed Continual Learning Mengqi Xue Haofei Zhang Mingli Song Mingli Song CLL 32 42 0 22 Mar 2022
Continual Sequence Generation with Adaptive Compositional Modules Yanzhe Zhang Xuezhi Wang Diyi Yang KELM CLL 51 42 0 20 Mar 2022
$Meta-X$_{NLG}$: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation$ Meta-X $_{NLG}$ : A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation Kaushal Kumar Maurya M. Desarkar 39 8 0 19 Mar 2022