Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 981 papers shown

Title
Legal and Political Stance Detection of SCOTUS Language Noah Bergam Emily Allaway Kathleen McKeown AILaw ELM 32 6 0 21 Nov 2022
AF Adapter: Continual Pretraining for Building Chinese Biomedical Language Model Yongyu Yan Kui Xue Xiaoming Shi Qi Ye Jingping Liu Tong Ruan CLL 47 1 0 21 Nov 2022
UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition Guimin Hu Ting-En Lin Yi Zhao Guangming Lu Yuchuan Wu Yongbin Li 39 111 0 21 Nov 2022
CL-CrossVQA: A Continual Learning Benchmark for Cross-Domain Visual Question Answering Yao Zhang Haokun Chen A. Frikha Yezi Yang Denis Krompass Gengyuan Zhang Jindong Gu Volker Tresp VLM LRM 16 7 0 19 Nov 2022
Scaling Native Language Identification with Transformer Adapters Ahmet Uluslu Gerold Schneider 32 6 0 18 Nov 2022
ConStruct-VL: Data-Free Continual Structured VL Concepts Learning James Smith Paola Cascante-Bonilla Assaf Arbelle Donghyun Kim Yikang Shen David D. Cox Diyi Yang Z. Kira Rogerio Feris Leonid Karlinsky VLM 49 20 0 17 Nov 2022
Cross-Modal Adapter for Text-Video Retrieval Haojun Jiang Jianke Zhang Rui Huang Chunjiang Ge Zanlin Ni Jiwen Lu Jie Zhou S. Song Gao Huang 53 37 0 17 Nov 2022
Prompt Tuning for Parameter-efficient Medical Image Segmentation Marc Fischer Alexander Bartler Bin Yang SSeg 26 19 0 16 Nov 2022
Towards Robust Low-Resource Fine-Tuning with Multi-View Compressed Representations Linlin Liu Xingxuan Li Megh Thakkar Xin Li Chenyu You Luo Si Lidong Bing 32 2 0 16 Nov 2022
Parameter-Efficient Tuning on Layer Normalization for Pre-trained Language Models Wang Qi Yu-Ping Ruan Y. Zuo Taihao Li 32 18 0 16 Nov 2022
On the Compositional Generalization Gap of In-Context Learning Arian Hosseini Ankit Vani Dzmitry Bahdanau Alessandro Sordoni Rameswar Panda 34 24 0 15 Nov 2022
Introducing Semantics into Speech Encoders Derek Xu Shuyan Dong Changhan Wang Suyoun Kim Zhaojiang Lin ... Alexei Baevski Guan-Ting Lin Hung-yi Lee Yizhou Sun Wei Wang SSL 38 3 0 15 Nov 2022
FedTune: A Deep Dive into Efficient Federated Fine-Tuning with Pre-trained Transformers Jinyu Chen Wenchao Xu Song Guo Junxiao Wang Jie Zhang Yining Qi FedML 37 32 0 15 Nov 2022
An Overview on Controllable Text Generation via Variational Auto-Encoders Haoqin Tu Yitong Li BDL 37 2 0 15 Nov 2022
Finding Skill Neurons in Pre-trained Transformer-based Language Models Xiaozhi Wang Kaiyue Wen Zhengyan Zhang Lei Hou Zhiyuan Liu Juanzi Li MILM MoE 34 51 0 14 Nov 2022
FPT: Improving Prompt Tuning Efficiency via Progressive Training Yufei Huang Yujia Qin Huadong Wang Yichun Yin Maosong Sun Zhiyuan Liu Qun Liu VLM LRM 37 6 0 13 Nov 2022
BERT-Deep CNN: State-of-the-Art for Sentiment Analysis of COVID-19 Tweets Javad Hassannataj Joloudari Sadiq Hussain M. Nematollahi Rouhollah Bagheri Fatemeh Fazl R. Alizadehsani Reza Lashgari Ashis Talukder 18 39 0 04 Nov 2022
Two-stage LLM Fine-tuning with Less Specialization and More Generalization Yihan Wang Si Si Daliang Li Michal Lukasik Felix X. Yu Cho-Jui Hsieh Inderjit S Dhillon Sanjiv Kumar 51 29 0 01 Nov 2022
Adapter-Based Extension of Multi-Speaker Text-to-Speech Model for New Speakers Cheng-Ping Hsieh Subhankar Ghosh Boris Ginsburg 48 18 0 01 Nov 2022
Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks Rochelle Choenni Dan Garrette Ekaterina Shutova 36 2 0 31 Oct 2022
Domain Curricula for Code-Switched MT at MixMT 2022 Lekan Raheem Maab Elrashid 34 1 0 31 Oct 2022
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning Yaqing Wang Sahaj Agarwal Subhabrata Mukherjee Xiaodong Liu Jing Gao Ahmed Hassan Awadallah Jianfeng Gao MoE 27 118 0 31 Oct 2022
GPS: Genetic Prompt Search for Efficient Few-shot Learning Hanwei Xu Yujun Chen Yulun Du Nan Shao Yanggang Wang Haiyu Li Zhilin Yang VLM 14 28 0 31 Oct 2022
Parameter-Efficient Tuning Makes a Good Classification Head Zhuoyi Yang Ming Ding Yanhui Guo Qingsong Lv Jie Tang VLM 65 14 0 30 Oct 2022
Differentiable Data Augmentation for Contrastive Sentence Representation Learning Tianduo Wang Wei Lu SSL 47 10 0 29 Oct 2022
Weight Averaging: A Simple Yet Effective Method to Overcome Catastrophic Forgetting in Automatic Speech Recognition Steven Vander Eeckt Hugo Van hamme CLL MoMe 67 14 0 27 Oct 2022
Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning Barun Patra Saksham Singhal Shaohan Huang Zewen Chi Li Dong Furu Wei Vishrav Chaudhary Xia Song 71 23 0 26 Oct 2022
Inducer-tuning: Connecting Prefix-tuning and Adapter-tuning Yifan Chen Devamanyu Hazarika Mahdi Namazifar Yang Liu Di Jin Dilek Z. Hakkani-Tür 34 3 0 26 Oct 2022
Learning Better Intent Representations for Financial Open Intent Classification Xianzhi Li Will Aitken Xiao-Dan Zhu Stephen W. Thomas AIFin 22 8 0 25 Oct 2022
Leveraging Open Data and Task Augmentation to Automated Behavioral Coding of Psychotherapy Conversations in Low-Resource Scenarios Zhuohao Chen Nikolaos Flemotomos Zac E. Imel David C. Atkins Shrikanth Narayanan 28 4 0 25 Oct 2022
Exploring Mode Connectivity for Pre-trained Language Models Yujia Qin Cheng Qian Jing Yi Weize Chen Yankai Lin Xu Han Zhiyuan Liu Maosong Sun Jie Zhou 34 20 0 25 Oct 2022
Evaluating Parameter Efficient Learning for Generation Peng Xu M. Patwary Shrimai Prabhumoye Virginia Adams R. Prenger Ming-Yu Liu Nayeon Lee Mohammad Shoeybi Bryan Catanzaro MoE 40 3 0 25 Oct 2022
Adapters for Enhanced Modeling of Multilingual Knowledge and Text Yifan Hou Wenxiang Jiao Mei-Jun Liu Carl Allen Zhaopeng Tu Mrinmaya Sachan 44 11 0 24 Oct 2022
Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning Jing Yi Weize Chen Yujia Qin Yankai Lin Ning Ding Xu Han Zhiyuan Liu Maosong Sun Jie Zhou 25 2 0 24 Oct 2022
NVIDIA FLARE: Federated Learning from Simulation to Real-World H. Roth Yan Cheng Yuhong Wen Isaac Yang Ziyue Xu ... Daguang Xu Nic Ma Prerna Dogra Mona G. Flores Andrew Feng FedML AI4CE 27 97 0 24 Oct 2022
Specializing Multi-domain NMT via Penalizing Low Mutual Information Jiyoung Lee H. Kim Hyun Chang Cho Edward Choi Cheonbok Park 21 4 0 24 Oct 2022
$m^4Adapter$ : Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adapter Wen Lai Alexandra Chronopoulou Alexander Fraser 31 3 0 21 Oct 2022
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts Xiangyang Liu Tianxiang Sun Xuanjing Huang Xipeng Qiu VLM 36 27 0 20 Oct 2022
Disentangling Reasoning Capabilities from Language Models with Compositional Reasoning Transformers Wanjun Zhong Tingting Ma Jiahai Wang Jian Yin Tiejun Zhao Chin-Yew Lin Nan Duan LRM CoGe 38 2 0 20 Oct 2022
Towards Sustainable Self-supervised Learning Shanghua Gao Pan Zhou Mingg-Ming Cheng Shuicheng Yan CLL 58 7 0 20 Oct 2022
Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking Tim Baumgärtner Leonardo F. R. Ribeiro Nils Reimers Iryna Gurevych 39 6 0 19 Oct 2022
Revision Transformers: Instructing Language Models to Change their Values Felix Friedrich Wolfgang Stammer P. Schramowski Kristian Kersting KELM 38 6 0 19 Oct 2022
Continued Pretraining for Better Zero- and Few-Shot Promptability Zhaofeng Wu IV RobertL.Logan Pete Walsh Akshita Bhagia Dirk Groeneveld Sameer Singh Iz Beltagy VLM 44 12 0 19 Oct 2022
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning Shuo Xie Jiahao Qiu Ankita Pasad Li Du Qing Qu Hongyuan Mei 42 16 0 18 Oct 2022
Tiny-Attention Adapter: Contexts Are More Important Than the Number of Parameters Hongyu Zhao Hao Tan Hongyuan Mei MoE 44 16 0 18 Oct 2022
Using Bottleneck Adapters to Identify Cancer in Clinical Notes under Low-Resource Constraints Omid Rohanian Hannah Jauncey Mohammadmahdi Nouriborji Vinod Kumar Chauhan Bronner P. Gonccalves Christiana Kartsonaki Isaric Clinical Characterisation Group L. Merson David Clifton 32 7 0 17 Oct 2022
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning Dongze Lian Daquan Zhou Jiashi Feng Xinchao Wang 38 250 0 17 Oct 2022
SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning Tzu-hsun Feng Annie Dong Ching-Feng Yeh Shu-Wen Yang Tzu-Quan Lin ... Xuankai Chang Shinji Watanabe Abdel-rahman Mohamed Shang-Wen Li Hung-yi Lee ELM SSL 41 33 0 16 Oct 2022
$RedApt: An Adaptor for wav2vec 2 Encoding \\ Faster and Smaller Speech Translation without Quality Compromise$ RedApt: An Adaptor for wav2vec 2 Encoding \\ Faster and Smaller Speech Translation without Quality Compromise Jinming Zhao Haomiao Yang Gholamreza Haffari Ehsan Shareghi VLM 21 2 0 16 Oct 2022
BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation Tianxiang Sun Junliang He Xipeng Qiu Xuanjing Huang 29 44 0 14 Oct 2022