BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

18 June 2021

Papers citing "BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models"

50 / 819 papers shown

Title
Backdoor Learning for NLP: Recent Advances, Challenges, and Future Research Directions Marwan Omar SILM AAML 33 20 0 14 Feb 2023
Task-Specific Skill Localization in Fine-tuned Language Models A. Panigrahi Nikunj Saunshi Haoyu Zhao Sanjeev Arora MoMe 34 68 0 13 Feb 2023
Less is More: Selective Layer Finetuning with SubTuning Gal Kaplun Andrey Gurevich Tal Swisa Mazor David Shai Shalev-Shwartz Eran Malach 26 7 0 13 Feb 2023
Offsite-Tuning: Transfer Learning without Full Model Guangxuan Xiao Ji Lin Song Han 43 67 0 09 Feb 2023
Knowledge is a Region in Weight Space for Fine-tuned Language Models Almog Gueta Elad Venezian Colin Raffel Noam Slonim Yoav Katz Leshem Choshen 34 49 0 09 Feb 2023
UDApter -- Efficient Domain Adaptation Using Adapters Bhavitvya Malik Abhinav Ramesh Kashyap MingSung Kan Soujanya Poria 29 16 0 07 Feb 2023
AIM: Adapting Image Models for Efficient Video Action Recognition Taojiannan Yang Yi Zhu Yusheng Xie Aston Zhang Cheng Chen Mu Li ViT 58 144 0 06 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 31 47 0 02 Feb 2023
An Empirical Study on the Transferability of Transformer Modules in Parameter-Efficient Fine-Tuning Mohammad AkbarTajari S. Rajaee Mohammad Taher Pilehvar 11 2 0 01 Feb 2023
AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning Han Zhou Xingchen Wan Ivan Vulić Anna Korhonen 24 45 0 28 Jan 2023
Backward Compatibility During Data Updates by Weight Interpolation Raphael Schumann Elman Mansimov Yi-An Lai Nikolaos Pappas Xibin Gao Yi Zhang 19 4 0 25 Jan 2023
Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute Michiel de Jong Yury Zemlyanskiy Nicholas FitzGerald Joshua Ainslie Sumit Sanghai Fei Sha William W. Cohen RALM 32 16 0 25 Jan 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 23 3 0 24 Jan 2023
Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning Malte Ostendorff Georg Rehm CLIP VLM CLL 41 23 0 23 Jan 2023
Differentially Private Natural Language Models: Recent Advances and Future Directions Lijie Hu Ivan Habernal Lei Shen Di Wang AAML 30 18 0 22 Jan 2023
Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction Shaofei Cai Zihao Wang Xiaojian Ma Guy Van den Broeck Yitao Liang 50 40 0 21 Jan 2023
From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognition Chao-Han Huck Yang Bo-wen Li Yu Zhang Nanxin Chen Rohit Prabhavalkar Tara N. Sainath Trevor Strohman 19 28 0 19 Jan 2023
Parameter-Efficient Fine-Tuning Design Spaces Jiaao Chen Aston Zhang Xingjian Shi Mu Li Alexander J. Smola Diyi Yang 42 59 0 04 Jan 2023
SPT: Semi-Parametric Prompt Tuning for Multitask Prompted Learning M Saiful Bari Aston Zhang Shuai Zheng Xingjian Shi Yi Zhu Chenyu You Mu Li RALM VLM VPVLM LRM 48 5 0 21 Dec 2022
KronA: Parameter Efficient Tuning with Kronecker Adapter Ali Edalati Marzieh S. Tahaei I. Kobyzev V. Nia J. Clark Mehdi Rezagholizadeh 45 94 0 20 Dec 2022
When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods Zhuo Zhang Yuanhang Yang Yong Dai Lizhen Qu Zenglin Xu FedML 46 66 0 20 Dec 2022
A Natural Bias for Language Generation Models Clara Meister Wojciech Stokowiec Tiago Pimentel Lei Yu Laura Rimell A. Kuncoro MILM 33 6 0 19 Dec 2022
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting Zheng-Xin Yong Hailey Schoelkopf Niklas Muennighoff Alham Fikri Aji David Ifeoluwa Adelani ... Genta Indra Winata Stella Biderman Edward Raff Dragomir R. Radev Vassilina Nikoulina CLL VLM AI4CE LRM 35 81 0 19 Dec 2022
Decoder Tuning: Efficient Language Understanding as Decoding Ganqu Cui Wentao Li Ning Ding Longtao Huang Zhiyuan Liu Maosong Sun 21 6 0 16 Dec 2022
Vision Transformers are Parameter-Efficient Audio-Visual Learners Yan-Bo Lin Yi-Lin Sung Jie Lei Joey Tianyi Zhou Gedas Bertasius 34 73 0 15 Dec 2022
Federated Few-Shot Learning for Mobile NLP Dongqi Cai Shangguang Wang Yaozong Wu F. Lin Mengwei Xu FedML 13 12 0 12 Dec 2022
Parameter-Efficient Finetuning of Transformers for Source Code Shamil Ayupov Nadezhda Chirkova 22 17 0 12 Dec 2022
Federated Learning for Inference at Anytime and Anywhere Zicheng Liu Da Li Javier Fernandez-Marques Stefanos Laskaridis Yan Gao L. Dudziak Stan Z. Li S. Hu Timothy M. Hospedales FedML 32 5 0 08 Dec 2022
JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset Ruth-Ann Armstrong John Hewitt Christopher D. Manning 33 14 0 07 Dec 2022
Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learning Cheng-Hao Tu Zheda Mai Wei-Lun Chao 32 44 0 06 Dec 2022
FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer Shibo Jie Zhi-Hong Deng 28 126 0 06 Dec 2022
M-VADER: A Model for Diffusion with Multimodal Context Samuel Weinbach Marco Bellagente C. Eichenberg Andrew M. Dai R. Baldock Souradeep Nanda Bjorn Deiseroth Koen Oostermeijer H. Teufel Andres Felipe Cruz Salinas DiffM 35 11 0 06 Dec 2022
Improving Few-Shot Performance of Language Models via Nearest Neighbor Calibration Feng Nie Meixi Chen Zhirui Zhang Xuan Cheng 24 32 0 05 Dec 2022
Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning Shih-Cheng Huang Shi Wang Min-Han Shih Saurav Sahay Hung-yi Lee 42 0 0 02 Dec 2022
CHAPTER: Exploiting Convolutional Neural Network Adapters for Self-supervised Speech Models Zih-Ching Chen Yu-Shun Sung Hung-yi Lee 29 16 0 01 Dec 2022
Learning Label Modular Prompts for Text Classification in the Wild Hailin Chen Amrita Saha Chenyu You Steven C. H. Hoi OOD VLM 26 5 0 30 Nov 2022
SPARTAN: Sparse Hierarchical Memory for Parameter-Efficient Transformers Ameet Deshpande Md Arafat Sultan Anthony Ferritto Ashwin Kalyan Karthik Narasimhan Avirup Sil MoE 38 1 0 29 Nov 2022
On the Effectiveness of Parameter-Efficient Fine-Tuning Z. Fu Haoran Yang Anthony Man-Cho So Wai Lam Lidong Bing Nigel Collier 27 156 0 28 Nov 2022
Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization? Rishi Bommasani Kathleen A. Creel Ananya Kumar Dan Jurafsky Percy Liang 25 77 0 25 Nov 2022
Prototypical Fine-tuning: Towards Robust Performance Under Varying Data Sizes Yiqiao Jin Xiting Wang Y. Hao Yizhou Sun Xing Xie 38 11 0 24 Nov 2022
VoP: Text-Video Co-operative Prompt Tuning for Cross-Modal Retrieval Siteng Huang Biao Gong Yulin Pan Jianwen Jiang Yiliang Lv Yuyuan Li Donglin Wang VLM VPVLM 22 41 0 23 Nov 2022
Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans? Neeraj Varshney Man Luo Chitta Baral RALM 21 11 0 23 Nov 2022
ConStruct-VL: Data-Free Continual Structured VL Concepts Learning James Smith Paola Cascante-Bonilla Assaf Arbelle Donghyun Kim Yikang Shen David D. Cox Diyi Yang Z. Kira Rogerio Feris Leonid Karlinsky VLM 47 20 0 17 Nov 2022
Cross-Modal Adapter for Text-Video Retrieval Haojun Jiang Jianke Zhang Rui Huang Chunjiang Ge Zanlin Ni Jiwen Lu Jie Zhou S. Song Gao Huang 48 36 0 17 Nov 2022
Prompt Tuning for Parameter-efficient Medical Image Segmentation Marc Fischer Alexander Bartler Bin Yang SSeg 24 18 0 16 Nov 2022
On Measuring the Intrinsic Few-Shot Hardness of Datasets Xinran Zhao Shikhar Murty Christopher D. Manning 11 5 0 16 Nov 2022
Differentially Private Optimizers Can Learn Adversarially Robust Models Yuan Zhang Zhiqi Bu 24 3 0 16 Nov 2022
Parameter-Efficient Tuning on Layer Normalization for Pre-trained Language Models Wang Qi Yu-Ping Ruan Y. Zuo Taihao Li 27 18 0 16 Nov 2022
Finding Skill Neurons in Pre-trained Transformer-based Language Models Xiaozhi Wang Kaiyue Wen Zhengyan Zhang Lei Hou Zhiyuan Liu Juanzi Li MILM MoE 27 50 0 14 Nov 2022
FPT: Improving Prompt Tuning Efficiency via Progressive Training Yufei Huang Yujia Qin Huadong Wang Yichun Yin Maosong Sun Zhiyuan Liu Qun Liu VLM LRM 35 6 0 13 Nov 2022