BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

18 June 2021

Papers citing "BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models"

50 / 819 papers shown

Title
Convolutional Bypasses Are Better Vision Transformer Adapters Shibo Jie Zhi-Hong Deng VPVLM 21 131 0 14 Jul 2022
Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation Zejiang Hou Julian Salazar George Polovets 30 14 0 07 Jul 2022
On-Device Training Under 256KB Memory Ji Lin Ligeng Zhu Wei-Ming Chen Wei-Chen Wang Chuang Gan Song Han MQ 33 197 0 30 Jun 2022
ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning Junting Pan Ziyi Lin Xiatian Zhu Jing Shao Hongsheng Li 27 191 0 27 Jun 2022
Sparse Structure Search for Parameter-Efficient Tuning Shengding Hu Zhen Zhang Ning Ding Yadao Wang Yasheng Wang Zhiyuan Liu Maosong Sun 37 16 0 15 Jun 2022
LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning Yi-Lin Sung Jaemin Cho Joey Tianyi Zhou VLM 21 237 0 13 Jun 2022
Neural Prompt Search Yuanhan Zhang Kaiyang Zhou Ziwei Liu VPVLM VLM 46 144 0 09 Jun 2022
Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks Lukas Hauzenberger Shahed Masoudian Deepak Kumar Markus Schedl Navid Rekabsaz 30 17 0 30 May 2022
AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition Shoufa Chen Chongjian Ge Zhan Tong Jiangliu Wang Yibing Song Jue Wang Ping Luo 152 639 0 26 May 2022
Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation Tu Vu Aditya Barua Brian Lester Daniel Cer Mohit Iyyer Noah Constant CLL 23 64 0 25 May 2022
Know Where You're Going: Meta-Learning for Parameter-Efficient Fine-Tuning Mozhdeh Gheini Xuezhe Ma Jonathan May 44 5 0 25 May 2022
Enhancing Continual Learning with Global Prototypes: Counteracting Negative Representation Drift Xueying Bai Jinghuan Shang Yifan Sun Niranjan Balasubramanian CLL 35 1 0 24 May 2022
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 130 100 0 24 May 2022
Representation Projection Invariance Mitigates Representation Collapse Anastasia Razdaibiedina A. Khetan Zohar Karnin Daniel Khashabi Vishaal Kapoor V. Madan 40 5 0 23 May 2022
When does Parameter-Efficient Transfer Learning Work for Machine Translation? Ahmet Üstün Asa Cooper Stickland 37 7 0 23 May 2022
BBTv2: Towards a Gradient-Free Future with Large Language Models Tianxiang Sun Zhengfu He Hong Qian Yunhua Zhou Xuanjing Huang Xipeng Qiu 108 53 0 23 May 2022
Self-Supervised Speech Representation Learning: A Review Abdel-rahman Mohamed Hung-yi Lee Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin ... Shang-Wen Li Karen Livescu Lars Maaløe Tara N. Sainath Shinji Watanabe SSL AI4TS 137 350 0 21 May 2022
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning Haokun Liu Derek Tam Mohammed Muqeeth Jay Mohta Tenghao Huang Joey Tianyi Zhou Colin Raffel 47 851 0 11 May 2022
Empowering parameter-efficient transfer learning by recognizing the kernel structure in self-attention Yifan Chen Devamanyu Hazarika Mahdi Namazifar Yang Liu Di Jin Dilek Z. Hakkani-Tür 24 9 0 07 May 2022
Efficient Fine-Tuning of BERT Models on the Edge Danilo Vucetic Mohammadreza Tayaranian M. Ziaeefard J. Clark B. Meyer W. Gross 28 33 0 03 May 2022
AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks Chin-Lun Fu Zih-Ching Chen Yun-Ru Lee Hung-yi Lee 33 44 0 30 Apr 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 46 3,349 0 29 Apr 2022
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval Robert Litschko Ivan Vulić Goran Glavavs LRM 37 14 0 05 Apr 2022
Improved and Efficient Conversational Slot Labeling through Question Answering Gabor Fuisz Ivan Vulić Samuel Gibbons I. Casanueva Paweł Budzianowski 21 11 0 05 Apr 2022
PERFECT: Prompt-free and Efficient Few-shot Learning with Language Models Rabeeh Karimi Mahabadi Luke Zettlemoyer James Henderson Marzieh Saeidi Lambert Mathias Ves Stoyanov Majid Yazdani VLM 34 69 0 03 Apr 2022
Parameter-efficient Model Adaptation for Vision Transformers Xuehai He Chunyuan Li Pengchuan Zhang Jianwei Yang Qing Guo 30 84 0 29 Mar 2022
Few-Shot Learning with Siamese Networks and Label Tuning Thomas Müller Guillermo Pérez-Torró Marc Franco-Salvador VLM 28 38 0 28 Mar 2022
A Scalable Model Specialization Framework for Training and Inference using Submodels and its Application to Speech Model Personalization Fadi Biadsy Youzheng Chen Xia Zhang Oleg Rybakov Andrew Rosenberg Pedro J. Moreno 38 13 0 23 Mar 2022
Visual Prompt Tuning Menglin Jia Luming Tang Bor-Chun Chen Claire Cardie Serge Belongie Bharath Hariharan Ser-Nam Lim VLM VPVLM 40 1,533 0 23 Mar 2022
Hyperdecoders: Instance-specific decoders for multi-task NLP Hamish Ivison Matthew E. Peters AI4CE 31 20 0 15 Mar 2022
Modular and Parameter-Efficient Multimodal Fusion with Prompting Sheng Liang Mengjie Zhao Hinrich Schütze 38 42 0 15 Mar 2022
Uncertainty Estimation for Language Reward Models Adam Gleave G. Irving UQLM 39 31 0 14 Mar 2022
CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment Haoyu Song Li Dong Weinan Zhang Ting Liu Furu Wei VLM CLIP 33 136 0 14 Mar 2022
Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models Ning Ding Yujia Qin Guang Yang Fu Wei Zonghan Yang ... Jianfei Chen Yang Liu Jie Tang Juan Li Maosong Sun 32 196 0 14 Mar 2022
Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models Shengnan An Yifei Li Zeqi Lin Qian Liu Bei Chen Qiang Fu Weizhu Chen Nanning Zheng Jian-Guang Lou VLM AAML 42 40 0 07 Mar 2022
Unfreeze with Care: Space-Efficient Fine-Tuning of Semantic Parsing Models Weiqi Sun Haidar Khan Nicolas Guenon des Mesnards M. Rubino Konstantine Arkoudas 33 5 0 05 Mar 2022
Controlling the Focus of Pretrained Language Generation Models Jiabao Ji Yoon Kim James R. Glass Tianxing He 38 5 0 02 Mar 2022
HyperPrompt: Prompt-based Task-Conditioning of Transformers Yun He H. Zheng Yi Tay Jai Gupta Yu Du ... Yaguang Li Zhaoji Chen Donald Metzler Heng-Tze Cheng Ed H. Chi LRM VLM 26 84 0 01 Mar 2022
SGPT: GPT Sentence Embeddings for Semantic Search Niklas Muennighoff RALM 35 176 0 17 Feb 2022
Revisiting Parameter-Efficient Tuning: Are We Really There Yet? Guanzheng Chen Fangyu Liu Zaiqiao Meng Shangsong Liang 26 88 0 16 Feb 2022
EdgeFormer: A Parameter-Efficient Transformer for On-Device Seq2seq Generation Tao Ge Si-Qing Chen Furu Wei MoE 32 21 0 16 Feb 2022
Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings Malte Ostendorff Nils Rethmeier Isabelle Augenstein Bela Gipp Georg Rehm 21 73 0 14 Feb 2022
Context-Tuning: Learning Contextualized Prompts for Natural Language Generation Tianyi Tang Junyi Li Wayne Xin Zhao Ji-Rong Wen 33 15 0 21 Jan 2022
Efficient Hierarchical Domain Adaptation for Pretrained Language Models Alexandra Chronopoulou Matthew E. Peters Jesse Dodge 33 42 0 16 Dec 2021
VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks Yi-Lin Sung Jaemin Cho Joey Tianyi Zhou VLM VPVLM 37 343 0 13 Dec 2021
Intrinisic Gradient Compression for Federated Learning Luke Melas-Kyriazi Franklyn Wang FedML 23 3 0 05 Dec 2021
Emojich -- zero-shot emoji generation using Russian language: a technical report Alex Shonenkov Daria Bakshandaeva Denis Dimitrov Aleks D. Nikolich VLM 32 5 0 04 Dec 2021
Training Neural Networks with Fixed Sparse Masks Yi-Lin Sung Varun Nair Colin Raffel FedML 32 197 0 18 Nov 2021
On Transferability of Prompt Tuning for Natural Language Processing Yusheng Su Xiaozhi Wang Yujia Qin Chi-Min Chan Yankai Lin ... Peng Li Juanzi Li Lei Hou Maosong Sun Jie Zhou AAML VLM 26 98 0 12 Nov 2021
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 83 1,035 0 01 Nov 2021