Federated Learning of Large Language Models with Parameter-Efficient
Prompt Tuning and Adaptive Optimization

Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization

23 October 2023

Victor S. Sheng

Papers citing "Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization"

15 / 15 papers shown

Title
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models Zheng Lin Yuxin Zhang Zhe Chen Zihan Fang Xianhao Chen Praneeth Vepakomma Wei Ni Jun Luo Yue Gao MoE 46 2 0 05 May 2025
Mitigating Group-Level Fairness Disparities in Federated Visual Language Models Chaomeng Chen Zitong Yu Jin Song Dong Sen Su L. Shen Shutao Xia Xiaochun Cao FedML VLM 166 0 0 03 May 2025
Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptation Yong Li Bo Liu Sheng Huang Zhe Zhang Xiaotong Yuan Richang Hong 46 0 0 31 Mar 2025
Selective Aggregation for Low-Rank Adaptation in Federated Learning Pengxin Guo Shuang Zeng Y. Wang Huijie Fan Feifei Wang Liangqiong Qu FedML 44 10 0 02 Oct 2024
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 42 43 0 09 Jul 2024
Save It All: Enabling Full Parameter Tuning for Federated Large Language Models via Cycle Block Gradient Descent Lin Wang Zhichao Wang Xiaoying Tang 45 1 0 17 Jun 2024
Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security Youyang Qu Ming Liu Tianqing Zhu Longxiang Gao Shui Yu Wanlei Zhou MU FedML 65 2 0 14 Jun 2024
Personalized Collaborative Fine-Tuning for On-Device Large Language Models Nicolas Wagner Dongyang Fan Martin Jaggi 39 3 0 15 Apr 2024
On Mask-based Image Set Desensitization with Recognition Support Qilong Li Ji Liu Yifan Sun Chongsheng Zhang Dejing Dou CVBM 28 3 0 14 Dec 2023
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 40 32 0 11 Dec 2023
FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Update Ji Liu Juncheng Jia Tianshi Che Chao Huo Jiaxiang Ren Yang Zhou H. Dai Dejing Dou 24 32 0 10 Dec 2023
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 130 100 0 24 May 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,661 0 15 Oct 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,858 0 18 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,984 0 20 Apr 2018