Federated Learning of Large Language Models with Parameter-Efficient
Prompt Tuning and Adaptive Optimization

Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization

23 October 2023

Victor S. Sheng

Papers citing "Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization"

17 / 17 papers shown

Title
SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignment Wenqiao Zhu Ji Liu Lulu Wang Jun Wu Yulun Zhang 16 0 0 18 May 2025
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models Zheng Lin Yuxin Zhang Zhe Chen Zihan Fang Xianhao Chen Praneeth Vepakomma Wei Ni Jun Luo Yue Gao MoE 49 2 0 05 May 2025
Mitigating Group-Level Fairness Disparities in Federated Visual Language Models Chaomeng Chen Zitong Yu Jin Song Dong Sen Su Linlin Shen Shutao Xia Xiaochun Cao FedML VLM 220 0 0 03 May 2025
Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptation Yong Li Bo Liu Sheng Huang Zhe Zhang Xiaotong Yuan Richang Hong 51 0 0 31 Mar 2025
Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architectures Yicheng Zhang Zhen Qin Zhaomin Wu Jian Hou Shuiguang Deng 85 2 0 28 Nov 2024
Selective Aggregation for Low-Rank Adaptation in Federated Learning Pengxin Guo Shuang Zeng Y. Wang Huijie Fan Feifei Wang Liangqiong Qu FedML 50 11 0 02 Oct 2024
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 45 43 0 09 Jul 2024
Save It All: Enabling Full Parameter Tuning for Federated Large Language Models via Cycle Block Gradient Descent Lin Wang Zhichao Wang Xiaoying Tang 49 1 0 17 Jun 2024
Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security Youyang Qu Ming Liu Tianqing Zhu Longxiang Gao Shui Yu Wanlei Zhou MU FedML 68 2 0 14 Jun 2024
Personalized Collaborative Fine-Tuning for On-Device Large Language Models Nicolas Wagner Dongyang Fan Martin Jaggi 41 3 0 15 Apr 2024
On Mask-based Image Set Desensitization with Recognition Support Qilong Li Ji Liu Yifan Sun Chongsheng Zhang Dejing Dou CVBM 30 3 0 14 Dec 2023
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 45 32 0 11 Dec 2023
FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Update Ji Liu Juncheng Jia Tianshi Che Chao Huo Jiaxiang Ren Yang Zhou H. Dai Dejing Dou 26 32 0 10 Dec 2023
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 130 100 0 24 May 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 218 1,663 0 15 Oct 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,872 0 18 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 304 6,996 0 20 Apr 2018