BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

18 June 2021

Papers citing "BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models"

50 / 821 papers shown

Title
Data-Centric Foundation Models in Computational Healthcare: A Survey Yunkun Zhang Jin Gao Zheling Tan Lingfeng Zhou Kexin Ding Mu Zhou Shaoting Zhang Dequan Wang AI4CE 45 22 0 04 Jan 2024
Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models Terry Yue Zhuo A. Zebaze Nitchakarn Suppattarachai Leandro von Werra H. D. Vries Qian Liu Niklas Muennighoff ALM 41 15 0 01 Jan 2024
Differentially Private Low-Rank Adaptation of Large Language Model Using Federated Learning Xiao-Yang Liu Rongyi Zhu Daochen Zha Jiechao Gao Shan Zhong Matt White Meikang Qiu 29 15 0 29 Dec 2023
A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Software Engineering Tasks Wentao Zou Qi Li Jidong Ge Chuanyi Li Xiaoyu Shen LiGuo Huang Bin Luo 34 5 0 25 Dec 2023
A Split-and-Privatize Framework for Large Language Model Fine-Tuning Xicong Shen Yang Liu Huiqi Liu Jue Hong Bing Duan Zirui Huang Yunlong Mao Ye Wu Di Wu 70 12 0 25 Dec 2023
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel Sebastian Pokutta VLM 52 10 0 23 Dec 2023
Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Models Alan Chan Ben Bucknall Herbie Bradley David M. Krueger 14 6 0 22 Dec 2023
ElasticTrainer: Speeding Up On-Device Training with Runtime Elastic Tensor Selection Kai Huang Boyuan Yang Wei Gao 32 18 0 21 Dec 2023
PPEA-Depth: Progressive Parameter-Efficient Adaptation for Self-Supervised Monocular Depth Estimation Yue-Jiang Dong Yuanchen Guo Ying-Tian Liu Fang-Lue Zhang Song-Hai Zhang MDE 43 4 0 20 Dec 2023
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Lingling Xu Haoran Xie S. J. Qin Xiaohui Tao F. Wang 55 137 0 19 Dec 2023
Sparse is Enough in Fine-tuning Pre-trained Large Language Models Weixi Song Z. Li Lefei Zhang Hai Zhao Bo Du VLM 29 7 0 19 Dec 2023
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 35 1 0 18 Dec 2023
When Parameter-efficient Tuning Meets General-purpose Vision-language Models Yihang Zhai Haixin Wang Jianlong Chang Xinlong Yang Jinan Sun Shikun Zhang Qi Tian VLM MLLM 27 1 0 16 Dec 2023
SPT: Fine-Tuning Transformer-based Language Models Efficiently with Sparsification Yuntao Gui Xiao Yan Peiqi Yin Han Yang James Cheng 43 2 0 16 Dec 2023
Gradient-based Parameter Selection for Efficient Fine-Tuning Zhi Zhang Qizhe Zhang Zijun Gao Renrui Zhang Ekaterina Shutova Shiji Zhou Shanghang Zhang 33 15 0 15 Dec 2023
Context-PEFT: Efficient Multi-Modal, Multi-Task Fine-Tuning Avelina Asada Hadji-Kyriacou Ognjen Arandjelović 35 0 0 14 Dec 2023
VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding Yi Xin Junlong Du Qiang Wang Zhiwen Lin Ke Yan VPVLM 92 49 0 14 Dec 2023
MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learning Yi Xin Junlong Du Qiang Wang Ke Yan Shouhong Ding VLM 47 46 0 14 Dec 2023
EZ-CLIP: Efficient Zeroshot Video Action Recognition Shahzad Ahmad S. Chanda Yogesh S Rawat VLM 36 7 0 13 Dec 2023
READ-PVLA: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language Modeling Thong Nguyen Xiaobao Wu Xinshuai Dong Khoi M. Le Zhiyuan Hu Cong-Duy Nguyen See-Kiong Ng Anh Tuan Luu 36 2 0 12 Dec 2023
Mutual Enhancement of Large and Small Language Models with Cross-Silo Knowledge Transfer Yongheng Deng Ziqing Qiao Ju Ren Yang Liu Yaoxue Zhang 30 11 0 10 Dec 2023
Initialization Matters for Adversarial Transfer Learning Andong Hua Jindong Gu Zhiyu Xue Nicholas Carlini Eric Wong Yao Qin AAML 32 6 0 10 Dec 2023
Batched Low-Rank Adaptation of Foundation Models Yeming Wen Swarat Chaudhuri OffRL 29 19 0 09 Dec 2023
Stronger, Fewer, & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentation Zhixiang Wei Lin Chen Yi Jin Xiaoxiao Ma Tianle Liu Pengyang Lin Ben Wang H. Chen Jinjin Zheng 45 42 0 07 Dec 2023
MoSA: Mixture of Sparse Adapters for Visual Efficient Tuning Qizhe Zhang Bocheng Zou Ruichuan An Jiaming Liu Shanghang Zhang MoE 29 2 0 05 Dec 2023
Clustering Pseudo Language Family in Multilingual Translation Models with Fisher Information Matrix Xinyu Ma Xuebo Liu Min Zhang 45 1 0 05 Dec 2023
Towards General Purpose Vision Foundation Models for Medical Image Analysis: An Experimental Study of DINOv2 on Radiology Benchmarks Mohammed Baharoon Waseem Qureshi J. Ouyang Yanwu Xu Abdulrhman Aljouie Wei Peng MedIm AI4CE 51 8 0 04 Dec 2023
The Efficiency Spectrum of Large Language Models: An Algorithmic Survey Tianyu Ding Tianyi Chen Haidong Zhu Jiachen Jiang Yiqi Zhong Jinxin Zhou Guangzhi Wang Zhihui Zhu Ilya Zharkov Luming Liang 29 22 0 01 Dec 2023
Grounding Foundation Models through Federated Transfer Learning: A General Framework Yan Kang Tao Fan Hanlin Gu Xiaojin Zhang Lixin Fan Qiang Yang AI4CE 68 19 0 29 Nov 2023
Efficient Stitchable Task Adaptation Haoyu He Zizheng Pan Jing Liu Jianfei Cai Bohan Zhuang 34 3 0 29 Nov 2023
End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames Shuming Liu Chen-Da Liu-Zhang Chen Zhao Guohao Li 38 25 0 28 Nov 2023
BIM: Block-Wise Self-Supervised Learning with Masked Image Modeling Yixuan Luo Mengye Ren Sai Qian Zhang 28 0 0 28 Nov 2023
Parameter Efficient Fine-tuning via Cross Block Orchestration for Segment Anything Model Zelin Peng Zhengqin Xu Zhilin Zeng Lingxi Xie Qi Tian Wei Shen 22 10 0 28 Nov 2023
A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA Damjan Kalajdzievski ALM 33 80 0 28 Nov 2023
Side4Video: Spatial-Temporal Side Network for Memory-Efficient Image-to-Video Transfer Learning Huanjin Yao Wenhao Wu Zhiheng Li VLM 95 9 0 27 Nov 2023
Adapter is All You Need for Tuning Visual Tasks Dongshuo Yin Leiyi Hu Bin Li Youqun Zhang 18 15 0 25 Nov 2023
PrivateLoRA For Efficient Privacy Preserving LLM Yiming Wang Yu Lin Xiaodong Zeng Guannan Zhang 66 11 0 23 Nov 2023
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization Prateek Yadav Leshem Choshen Colin Raffel Mohit Bansal 32 13 0 22 Nov 2023
Towards Better Parameter-Efficient Fine-Tuning for Large Language Models: A Position Paper Chengyu Wang Junbing Yan Wei Zhang Jun Huang ALM 42 3 0 22 Nov 2023
Descriptor and Word Soups: Overcoming the Parameter Efficiency Accuracy Tradeoff for Out-of-Distribution Few-shot Learning Christopher Liao Theodoros Tsiligkaridis Brian Kulis OODD 49 5 0 21 Nov 2023
Applications of Large Scale Foundation Models for Autonomous Driving Yu Huang Yue Chen Zhu Li ELM AI4CE LRM ALM LM&Ro 61 15 0 20 Nov 2023
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning Han Guo P. Greengard Eric P. Xing Yoon Kim MQ 38 44 0 20 Nov 2023
Zero redundancy distributed learning with differential privacy Zhiqi Bu Justin Chiu Ruixuan Liu Sheng Zha George Karypis 51 8 0 20 Nov 2023
Sparse Low-rank Adaptation of Pre-trained Language Models Ning Ding Xingtai Lv Qiaosen Wang Yulin Chen Bowen Zhou Zhiyuan Liu Maosong Sun 27 55 0 20 Nov 2023
MultiLoRA: Democratizing LoRA for Better Multi-Task Learning Yiming Wang Yu Lin Xiaodong Zeng Guannan Zhang MoMe 47 20 0 20 Nov 2023
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning Clifton A. Poth Hannah Sterz Indraneil Paul Sukannya Purkayastha Leon Arne Engländer Timo Imhof Ivan Vulić Sebastian Ruder Iryna Gurevych Jonas Pfeiffer 32 45 0 18 Nov 2023
A Framework for Monitoring and Retraining Language Models in Real-World Applications Jaykumar Kasundra Claudia Schulz Melicaalsadat Mirsafian Stavroula Skylaki OffRL LRM 34 1 0 16 Nov 2023
Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying Adithya Renduchintala Tugrul Konuk Oleksii Kuchaiev MoMe 41 42 0 16 Nov 2023
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 33 27 0 15 Nov 2023
Tunable Soft Prompts are Messengers in Federated Learning Chenhe Dong Yuexiang Xie Bolin Ding Ying Shen Yaliang Li FedML 46 7 0 12 Nov 2023