DEeR: Deviation Eliminating and Noise Regulating for Privacy-preserving Federated Low-rank Adaptation

16 October 2024

Meilu Zhu

Axiu Mao

Jun Liu

Yixuan Yuan

ArXiv (abs)PDF HTML Github (4★)

Papers citing "DEeR: Deviation Eliminating and Noise Regulating for Privacy-preserving Federated Low-rank Adaptation"

26 / 26 papers shown

Title
BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs Sheng Zhang Yanbo Xu Naoto Usuyama Hanwen Xu J. Bagga ... Carlo Bifulco M. Lungren Tristan Naumann Sheng Wang Hoifung Poon LM&MA MedIm 229 233 0 10 Jan 2025
Federated LoRA with Sparse Communication Kevin Kuo Arian Raje Kousik Rajesh Virginia Smith 96 11 0 07 Jun 2024
SPD-CFL: Stepwise Parameter Dropout for Efficient Continual Federated Learning Yuning Yang H. Yu Tianrun Gao Tianrun Gao Xiaohong Liu Xiaodong Xu Ping Zhang Guangyu Wang 73 6 0 15 May 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation Jie Xu Karthikeyan P. Saravanan Rogier van Dalen Haaris Mehmood David Tuckey Mete Ozay 158 8 0 10 May 2024
FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decomposition Yuxuan Yan Qianqian Yang Shunpu Tang Zhiguo Shi 84 18 0 29 Apr 2024
LoRA Dropout as a Sparsity Regularizer for Overfitting Control Yang Lin Xinyu Ma Xu Chu Yujie Jin Zhibang Yang Yasha Wang Hong-yan Mei 90 26 0 15 Apr 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 257 398 0 21 Mar 2024
Improving LoRA in Privacy-preserving Federated Learning Youbang Sun Zitao Li Yaliang Li Bolin Ding 84 81 0 18 Mar 2024
Convolution Meets LoRA: Parameter Efficient Finetuning for Segment Anything Model Zihan Zhong Zhiqiang Tang Tong He Haoyang Fang Chun Yuan 102 48 0 31 Jan 2024
Communication-Efficient Personalized Federated Learning for Speech-to-Text Tasks Yichao Du Zhirui Zhang Linan Yue Xu Huang Yuqing Zhang Tong Xu Linli Xu Enhong Chen FedML 128 7 0 18 Jan 2024
Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models Yae Jee Cho Luyang Liu Zheng Xu Aldi Fahrezi Gauri Joshi 71 67 0 12 Jan 2024
When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications Qidong Liu Xian Wu Xiangyu Zhao Yuanshao Zhu Derong Xu Feng Tian Yefeng Zheng MoE 96 72 0 21 Oct 2023
Mitigating the Alignment Tax of RLHF Yong Lin Hangyu Lin Wei Xiong Shizhe Diao Zeming Zheng ... Han Zhao Nan Jiang Heng Ji Yuan Yao Tong Zhang MoMe CLL 87 80 0 12 Sep 2023
Visual Prompt Tuning Menglin Jia Luming Tang Bor-Chun Chen Claire Cardie Serge Belongie Bharath Hariharan Ser-Nam Lim VLM VPVLM 167 1,645 0 23 Mar 2022
VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks Yi-Lin Sung Jaemin Cho Joey Tianyi Zhou VLM VPVLM 112 356 0 13 Dec 2021
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 513 2,422 0 02 Sep 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 502 10,526 0 17 Jun 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 1.0K 29,926 0 26 Feb 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 252 4,305 0 01 Jan 2021
PraNet: Parallel Reverse Attention Network for Polyp Segmentation Deng-Ping Fan Ge-Peng Ji Tao Zhou Geng Chen Huazhu Fu Jianbing Shen Ling Shao ODL 235 1,175 0 13 Jun 2020
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 275 6,294 0 10 Dec 2019
Kvasir-SEG: A Segmented Polyp Dataset Debesh Jha P. Smedsrud Michael A. Riegler Pål Halvorsen Thomas de Lange Dag Johansen Haavard D. Johansen 188 1,178 0 16 Nov 2019
Non-convex Optimization for Machine Learning Prateek Jain Purushottam Kar 174 486 0 21 Dec 2017
Deep Learning with Differential Privacy Martín Abadi Andy Chu Ian Goodfellow H. B. McMahan Ilya Mironov Kunal Talwar Li Zhang FedML SyDa 218 6,172 0 01 Jul 2016
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 412 17,615 0 17 Feb 2016
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 355 18,654 0 06 Feb 2015