BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

18 June 2021

Papers citing "BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models"

50 / 819 papers shown

Title
Data Adaptive Traceback for Vision-Language Foundation Models in Image Classification Wenshuo Peng Kaipeng Zhang Yue Yang Hao Zhang Ping Luo VLM 29 2 0 11 Jul 2024
Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software Dezhi Ran Mengzhou Wu Wei Yang Tao Xie AI4CE 39 1 0 11 Jul 2024
ROSA: Random Subspace Adaptation for Efficient Fine-Tuning Marawan Gamal Abdel Hameed Aristides Milios Siva Reddy Guillaume Rabusseau CLL 32 3 0 10 Jul 2024
SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning Haiwen Diao Bo Wan Xu Jia Yunzhi Zhuge Ying Zhang Huchuan Lu Long Chen VLM 50 4 0 10 Jul 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard F. Fleuret MoE 67 6 0 10 Jul 2024
Parameter-Efficient and Memory-Efficient Tuning for Vision Transformer: A Disentangled Approach Taolin Zhang Jiawang Bai Zhihe Lu Dongze Lian Genping Wang Xinchao Wang Shu-Tao Xia 41 4 0 09 Jul 2024
Rethinking Image-to-Video Adaptation: An Object-centric Perspective Rui Qian Shuangrui Ding Dahua Lin OCL 52 1 0 09 Jul 2024
CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community Yan Liu Bin Guo Nuo Li Yasan Ding Zhouyangzi Zhang Zhiwen Yu 50 1 0 09 Jul 2024
Artificial Intuition: Efficient Classification of Scientific Abstracts Harsh Sakhrani Naseela Pervez Anirudh Ravi Kumar Fred Morstatter Alexandra Graddy-Reed Andrea Belz 23 0 0 08 Jul 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 54 25 0 08 Jul 2024
ASRRL-TTS: Agile Speaker Representation Reinforcement Learning for Text-to-Speech Speaker Adaptation Ruibo Fu Xin Qi Zhengqi Wen Jianhua Tao Tao Wang ... Xiaopeng Wang Shuchen Shi Yukun Liu Xuefei Liu Shuai Zhang 54 0 0 07 Jul 2024
See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition Chongjie Si Xiaokang Yang Wei Shen 45 5 0 07 Jul 2024
Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models Yuyan Chen Qiang Fu Ge Fan Lun Du Jian-Guang Lou Shi Han Dongmei Zhang Zhixu Li Yanghua Xiao MoE 29 16 0 04 Jul 2024
Lateralization LoRA: Interleaved Instruction Tuning with Modality-Specialized Adaptations Zhiyang Xu Minqian Liu Ying Shen Joy Rimchala Jiaxin Zhang Qifan Wang Yu Cheng Lifu Huang VLM 39 2 0 04 Jul 2024
ASteISR: Adapting Single Image Super-resolution Pre-trained Model for Efficient Stereo Image Super-resolution Yuanbo Zhou Yuyang Xue Wei Deng Xinlin Zhang Qinquan Gao Tong Tong 58 0 0 04 Jul 2024
SAFT: Towards Out-of-Distribution Generalization in Fine-Tuning Bac Nguyen Stefan Uhlich Fabien Cardinaux Lukas Mauch Marzieh Edraki Aaron Courville OODD CLL VLM 57 3 0 03 Jul 2024
Increasing Model Capacity for Free: A Simple Strategy for Parameter Efficient Fine-tuning Haobo Song Hao Zhao Soumajit Majumder Tao Lin 30 3 0 01 Jul 2024
CPT: Consistent Proxy Tuning for Black-box Optimization Yuanyang He Zitong Huang Xinxing Xu Rick Siow Mong Goh Salman Khan W. Zuo Yong Liu Chun-Mei Feng 48 0 0 01 Jul 2024
M2QA: Multi-domain Multilingual Question Answering Leon Arne Engländer Hannah Sterz Clifton A. Poth Jonas Pfeiffer Ilia Kuznetsov Iryna Gurevych VLM 43 1 0 01 Jul 2024
Expressivity of Neural Networks with Random Weights and Learned Biases Ezekiel Williams Avery Hee-Woon Ryoo Thomas Jiralerspong Alexandre Payeur M. Perich Luca Mazzucato Guillaume Lajoie 38 2 0 01 Jul 2024
AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tuning Yifan Yang Kai Zhen Ershad Banijamal Athanasios Mouchtaris Zheng Zhang 42 8 0 26 Jun 2024
Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning Arijit Sehanobish Avinava Dubey Krzysztof Choromanski Somnath Basu Roy Chowdhury Deepali Jain Vikas Sindhwani Snigdha Chaturvedi ALM 43 1 0 25 Jun 2024
Minimal Interaction Edge Tuning: A New Paradigm for Visual Adaptation Ningyuan Tang Minghao Fu Jianxin Wu VLM 47 0 0 25 Jun 2024
D2LLM: Decomposed and Distilled Large Language Models for Semantic Search Zihan Liao Hang Yu Jianguo Li Jun Wang Wei Zhang 36 3 0 25 Jun 2024
Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation Yuchen Yang Yingdong Shi Cheems Wang Xiantong Zhen Yuxuan Shi Jun Xu 40 1 0 24 Jun 2024
xCOMET-lite: Bridging the Gap Between Efficiency and Quality in Learned MT Evaluation Metrics Daniil Larionov Mikhail Seleznyov Vasiliy Viskov Alexander Panchenko Steffen Eger 37 3 0 20 Jun 2024
Computation-Efficient Semi-Supervised Learning for ECG-based Cardiovascular Diseases Detection Rushuang Zhou Zijun Liu Lei A. Clifton David A. Clifton Kannie W. Y. Chan Yuan-Ting Zhang Yining Dong 31 0 0 20 Jun 2024
Towards Minimal Targeted Updates of Language Models with Targeted Negative Training Lily H. Zhang Rajesh Ranganath Arya Tafvizi 33 1 0 19 Jun 2024
Bayesian-LoRA: LoRA based Parameter Efficient Fine-Tuning using Optimal Quantization levels and Rank Values trough Differentiable Bayesian Gates Cristian Meo Ksenia Sycheva Anirudh Goyal Justin Dauwels MQ 29 4 0 18 Jun 2024
Synergizing Foundation Models and Federated Learning: A Survey Shenghui Li Fanghua Ye Meng Fang Jiaxu Zhao Yun-Hin Chan Edith C. -H. Ngai Thiemo Voigt AI4CE 57 5 0 18 Jun 2024
Mitigate Negative Transfer with Similarity Heuristic Lifelong Prompt Tuning Chenyuan Wu Gangwei Jiang Defu Lian CLL 31 0 0 18 Jun 2024
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion Chenghao Fan Zhenyi Lu Wei Wei Jie Tian Xiaoye Qu Dangyang Chen Yu Cheng MoMe 48 5 0 17 Jun 2024
Promoting Data and Model Privacy in Federated Learning through Quantized LoRA Jianhao Zhu Changze Lv Xiaohua Wang Muling Wu Wenhao Liu Tianlong Li Zixuan Ling Cenyuan Zhang Xiaoqing Zheng Xuanjing Huang 44 3 0 16 Jun 2024
ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation Yurun Song Junchen Zhao Ian G. Harris Sangeetha Abdu Jyothi 32 3 0 16 Jun 2024
Mixture-of-Subspaces in Low-Rank Adaptation Taiqiang Wu Jiahao Wang Zhe Zhao Ngai Wong 49 22 0 16 Jun 2024
Reflecting on the State of Rehearsal-free Continual Learning with Pretrained Models Lukas Thede Karsten Roth Olivier J. Hénaff Matthias Bethge Zeynep Akata CLL 47 5 0 13 Jun 2024
Adapters Strike Back Jan-Martin O. Steitz Stefan Roth 35 5 0 10 Jun 2024
A Parameter-efficient Language Extension Framework for Multilingual ASR Wei Liu Jingyong Hou Dong Yang Muyong Cao Tan Lee CLL 72 2 0 10 Jun 2024
Multi-Prompting Decoder Helps Better Language Understanding Zifeng Cheng Zhaoling Chen Zhiwei Jiang Yafeng Yin Shiping Ge Yuliang Liu Qing Gu AI4CE 50 1 0 10 Jun 2024
Diving into Underwater: Segment Anything Model Guided Underwater Salient Instance Segmentation and A Large-scale Dataset Shijie Lian Ziyi Zhang Hua Li Wenjie Li Laurence Tianruo Yang Sam Kwong Runmin Cong VLM 31 12 0 10 Jun 2024
Federated LoRA with Sparse Communication Kevin Kuo Arian Raje Kousik Rajesh Virginia Smith 46 8 0 07 Jun 2024
Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe Alicja Ziarko Albert Q. Jiang Bartosz Piotrowski Wenda Li M. Jamnik Piotr Miłoś 40 0 0 06 Jun 2024
Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision Minglei Li Peng Ye Yongqi Huang Lin Zhang Tao Chen Tong He Jiayuan Fan Wanli Ouyang MoE 36 4 0 05 Jun 2024
LoFiT: Localized Fine-tuning on LLM Representations Fangcong Yin Xi Ye Greg Durrett 38 13 0 03 Jun 2024
LOLA: LLM-Assisted Online Learning Algorithm for Content Experiments Zikun Ye Hema Yoganarasimhan Yufeng Zheng 44 2 0 03 Jun 2024
Diffusion Tuning: Transferring Diffusion Models via Chain of Forgetting Jincheng Zhong Xingzhuo Guo Jiaxiang Dong Mingsheng Long DiffM 40 2 0 02 Jun 2024
LIDAO: Towards Limited Interventions for Debiasing (Large) Language Models Tianci Liu Haoyu Wang Shiyang Wang Yu Cheng Jing Gao ALM 35 0 0 01 Jun 2024
A Survey on Large Language Models for Code Generation Juyong Jiang Fan Wang Jiasi Shen Sungju Kim Sunghun Kim 53 166 0 01 Jun 2024
Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining Yi Wang C. Albrecht Xiao Xiang Zhu 37 6 0 30 May 2024
RAP: Efficient Text-Video Retrieval with Sparse-and-Correlated Adapter Meng Cao Haoran Tang Jinfa Huang Peng Jin Can Zhang Ruyang Liu Long Chen Xiaodan Liang Li-ming Yuan Ge Li 101 11 0 29 May 2024