BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

18 June 2021

Papers citing "BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models"

50 / 823 papers shown

Title
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 35 27 0 15 Nov 2023
Tunable Soft Prompts are Messengers in Federated Learning Chenhe Dong Yuexiang Xie Bolin Ding Ying Shen Yaliang Li FedML 46 7 0 12 Nov 2023
Aggregate, Decompose, and Fine-Tune: A Simple Yet Effective Factor-Tuning Method for Vision Transformer Dongping Chen 41 3 0 12 Nov 2023
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization Weiyang Liu Zeju Qiu Yao Feng Yuliang Xiu Yuxuan Xue ... Songyou Peng Yandong Wen Michael J. Black Adrian Weller Bernhard Schölkopf 50 58 0 10 Nov 2023
Mini but Mighty: Finetuning ViTs with Mini Adapters Imad Eddine Marouf Enzo Tartaglione Stéphane Lathuilière 41 5 0 07 Nov 2023
Task-Agnostic Low-Rank Adapters for Unseen English Dialects Zedian Xiao William B. Held Yanchen Liu Diyi Yang 44 7 0 02 Nov 2023
Improving Prompt Tuning with Learned Prompting Layers Wei Zhu Ming Tan VLM 33 1 0 31 Oct 2023
Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone Zeyinzi Jiang Chaojie Mao Ziyuan Huang Ao Ma Yiliang Lv Yujun Shen Deli Zhao Jingren Zhou 38 15 0 30 Oct 2023
Fast Trainable Projection for Robust Fine-Tuning Junjiao Tian Yen-Cheng Liu James Seale Smith Z. Kira OOD 41 11 0 29 Oct 2023
Punica: Multi-Tenant LoRA Serving Lequn Chen Zihao Ye Yongji Wu Danyang Zhuo Luis Ceze Arvind Krishnamurthy 44 34 0 28 Oct 2023
Parameter-Efficient Methods for Metastases Detection from Clinical Notes Maede Ashofteh Barabadi Xiaodan Zhu Wai-Yip Chan Amber L. Simpson Richard K G Do 29 1 0 27 Oct 2023
PAC-tuning:Fine-tuning Pretrained Language Models with PAC-driven Perturbed Gradient Descent Guang-Da Liu Zhiyu Xue Xitong Zhang K. Johnson Rongrong Wang 33 5 0 26 Oct 2023
The Expressive Power of Low-Rank Adaptation Yuchen Zeng Kangwook Lee 41 51 0 26 Oct 2023
Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Models Raymond Li Gabriel Murray Giuseppe Carenini MoE 41 2 0 24 Oct 2023
Knowledge Editing for Large Language Models: A Survey Song Wang Yaochen Zhu Haochen Liu Zaiyi Zheng Chen Chen Wenlin Yao KELM 81 137 0 24 Oct 2023
Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules Chaojun Xiao Yuqi Luo Wenbin Zhang Pengle Zhang Xu Han ... Zhengyan Zhang Ruobing Xie Zhiyuan Liu Maosong Sun Jie Zhou 35 0 0 24 Oct 2023
CRaSh: Clustering, Removing, and Sharing Enhance Fine-tuning without Full Large Language Model Kaiyan Zhang Ning Ding Biqing Qi Ganqu Cui Xinwei Long Bowen Zhou 46 4 0 24 Oct 2023
A Communication Theory Perspective on Prompting Engineering Methods for Large Language Models Yuanfeng Song Yuanqin He Xuefang Zhao Hanlin Gu Di Jiang Haijun Yang Lixin Fan Qiang Yang 40 3 0 24 Oct 2023
Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attention Negar Foroutan Mohammadreza Banaei Karl Aberer Antoine Bosselut LRM 25 3 0 23 Oct 2023
Geographical Erasure in Language Generation Pola Schwöbel Jacek Golebiowski Michele Donini Cédric Archambeau Danish Pruthi 26 5 0 23 Oct 2023
Orthogonal Subspace Learning for Language Model Continual Learning Xiao Wang Tianze Chen Qiming Ge Han Xia Rong Bao Rui Zheng Qi Zhang Tao Gui Xuanjing Huang CLL 122 90 0 22 Oct 2023
Scalable Neural Network Kernels Arijit Sehanobish Krzysztof Choromanski Yunfan Zhao Kumar Avinava Dubey Valerii Likhosherstov 41 5 0 20 Oct 2023
RSAdapter: Adapting Multimodal Models for Remote Sensing Visual Question Answering Yuduo Wang Pedram Ghamisi 30 4 0 19 Oct 2023
Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understanding Jianing Wang Qiushi Sun Nuo Chen Chengyu Wang Jun Huang Ming Gao Xiang Li UQLM 44 3 0 19 Oct 2023
VeRA: Vector-based Random Matrix Adaptation D. J. Kopiczko Tijmen Blankevoort Yuki Markus Asano VLM 27 135 0 17 Oct 2023
Domain Generalization Using Large Pretrained Models with Mixture-of-Adapters Gyuseong Lee Wooseok Jang Jin Hyeon Kim Jaewoo Jung Seungryong Kim MoE OOD 30 3 0 17 Oct 2023
PELA: Learning Parameter-Efficient Models with Low-Rank Approximation Yangyang Guo Guangzhi Wang Mohan S. Kankanhalli 21 3 0 16 Oct 2023
Decomposed Prompt Tuning via Low-Rank Reparameterization Yao Xiao Lu Xu Jiaxi Li Wei Lu Xiaoli Li VLM 25 6 0 16 Oct 2023
Search-Adaptor: Embedding Customization for Information Retrieval Jinsung Yoon Sercan Ö. Arik Yanfei Chen Tomas Pfister 25 2 0 12 Oct 2023
Language Models are Universal Embedders Xin Zhang Zehan Li Yanzhao Zhang Dingkun Long Pengjun Xie Meishan Zhang Min Zhang KELM ELM 58 6 0 12 Oct 2023
Efficient Adaptation of Large Vision Transformer via Adapter Re-Composing Wei Dong Dawei Yan Zhijun Lin Peng Wang 27 21 0 10 Oct 2023
Hierarchical Side-Tuning for Vision Transformers Weifeng Lin Ziheng Wu Wentao Yang Mingxin Huang Jun Huang Lianwen Jin 31 8 0 09 Oct 2023
FairTune: Optimizing Parameter Efficient Fine Tuning for Fairness in Medical Image Analysis Raman Dutt Ondrej Bohdal Sotirios A. Tsaftaris Timothy M. Hospedales 26 14 0 08 Oct 2023
Randomized Sparse Neural Galerkin Schemes for Solving Evolution Equations with Deep Networks Jules Berman Benjamin Peherstorfer 26 14 0 07 Oct 2023
Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation Md Kaykobad Reza Ashley Prater-Bennette Ulugbek S. Kamilov 28 5 0 06 Oct 2023
Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! Xiangyu Qi Yi Zeng Tinghao Xie Pin-Yu Chen Ruoxi Jia Prateek Mittal Peter Henderson SILM 70 535 0 05 Oct 2023
Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models Yiwen Tang Ivan Tang Ray Gu Dong Wang Eric Zhang Bin Zhao Xuelong Li 3DPC 34 20 0 04 Oct 2023
NOLA: Compressing LoRA using Linear Combination of Random Basis Soroush Abbasi Koohpayegani K. Navaneet Parsa Nooralinejad Soheil Kolouri Hamed Pirsiavash 40 12 0 04 Oct 2023
Empirical Study of PEFT techniques for Winter Wheat Segmentation Mohamad Hasan Zahweh Hasan Nasrallah Mustafa Shukor G. Faour A. Ghandour 27 3 0 03 Oct 2023
FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models Jingwei Sun Ziyue Xu Hongxu Yin Dong Yang Daguang Xu Yiran Chen Holger R. Roth VLM 101 23 0 02 Oct 2023
ZeroI2V: Zero-Cost Adaptation of Pre-trained Transformers from Image to Video Xinhao Li Yuhan Zhu Limin Wang VLM 35 8 0 02 Oct 2023
ScaLearn: Simple and Highly Parameter-Efficient Task Transfer by Learning to Scale Markus Frohmann Carolin Holtermann Shahed Masoudian Anne Lauscher Navid Rekabsaz 47 2 0 02 Oct 2023
ELIP: Efficient Language-Image Pre-training with Fewer Vision Tokens Yangyang Guo Haoyu Zhang Yongkang Wong Liqiang Nie Mohan Kankanhalli VLM 30 3 0 28 Sep 2023
ModuLoRA: Finetuning 2-Bit LLMs on Consumer GPUs by Integrating with Modular Quantizers Junjie Yin Jiahao Dong Yingheng Wang Christopher De Sa Volodymyr Kuleshov MQ 33 4 0 28 Sep 2023
Low-rank Adaptation of Large Language Model Rescoring for Parameter-Efficient Speech Recognition Yu Yu Chao-Han Huck Yang J. Kolehmainen Prashanth Gurunath Shivakumar Yile Gu ... Denis Filimonov Shalini Ghosh A. Stolcke Ariya Rastrow I. Bulyko 40 8 0 26 Sep 2023
BLIP-Adapter: Parameter-Efficient Transfer Learning for Mobile Screenshot Captioning Ching-Yu Chiang I-Hua Chang Shih-Wei Liao 55 1 0 26 Sep 2023
ConPET: Continual Parameter-Efficient Tuning for Large Language Models Chenyan Song Xu Han Zheni Zeng Kuai Li Chen Chen Zhiyuan Liu Maosong Sun Taojiannan Yang CLL KELM 21 10 0 26 Sep 2023
Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation Kai Huang Hanyu Yin Heng Huang Wei Gao 33 11 0 22 Sep 2023
LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models Yukang Chen Shengju Qian Haotian Tang Xin Lai Zhijian Liu Song Han Jiaya Jia 61 153 0 21 Sep 2023
Sparsely Shared LoRA on Whisper for Child Speech Recognition W. Liu Ying Qin Zhiyuan Peng Tan Lee 27 17 0 21 Sep 2023