Towards Building the Federated GPT: Federated Instruction Tuning

9 May 2023

Jianyi Zhang

Saeed Vahidian

Yiran Chen

Papers citing "Towards Building the Federated GPT: Federated Instruction Tuning"

44 / 44 papers shown

Title
Dynamic Domain Information Modulation Algorithm for Multi-domain Sentiment Analysis Chunyi Yue Ang Li 34 0 0 10 May 2025
FedTDP: A Privacy-Preserving and Unified Framework for Trajectory Data Preparation via Federated Learning Zhihao Zeng Ziquan Fang Wei Shao Lu Chen Yunjun Gao FedML 51 0 0 08 May 2025
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models Zheng Lin Yuxin Zhang Zhe Chen Zihan Fang Xianhao Chen Praneeth Vepakomma Wei Ni Jun Luo Yue Gao MoE 49 2 0 05 May 2025
Federated Adapter on Foundation Models: An Out-Of-Distribution Approach Yiyuan Yang Guodong Long Dinesh Manocha Qinghua Lu Shanshan Ye Jing Jiang OODD 225 1 0 02 May 2025
Communication-Efficient Wireless Federated Fine-Tuning for Large-Scale AI Models Bumjun Kim Wan Choi 34 0 0 01 May 2025
A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning Jieming Bian Yuanzhe Peng Lei Wang Yin Huang Jie Xu FedML 65 0 0 29 Apr 2025
FedMerge: Federated Personalization via Model Merging Shutong Chen Tianyi Zhou Guodong Long Jing Jiang Chengqi Zhang FedML MoMe 49 0 0 09 Apr 2025
Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning Raghav Singhal Kaustubh Ponkshe Rohit Vartak Lav R. Varshney Praneeth Vepakomma FedML 79 1 0 24 Feb 2025
A General Pseudonymization Framework for Cloud-Based LLMs: Replacing Privacy Information in Controlled Text Generation Shilong Hou Ruilin Shang Zi Long Xianghua Fu Yin Chen 67 0 0 24 Feb 2025
Proactive Privacy Amnesia for Large Language Models: Safeguarding PII with Negligible Impact on Model Utility Martin Kuo Jingyang Zhang Jianyi Zhang Minxue Tang Louis DiValentin ... William Chen Amin Hass Tianlong Chen Yuxiao Chen Yiming Li MU KELM 51 2 0 24 Feb 2025
Ten Challenging Problems in Federated Foundation Models Tao Fan Hanlin Gu Xuemei Cao Chee Seng Chan Qian Chen ... Y. Zhang Xiaojin Zhang Zhenzhe Zheng Lixin Fan Qiang Yang FedML 89 4 0 14 Feb 2025
Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions Hao Du Shang Liu Lele Zheng Yang Cao Atsuyoshi Nakamura Lei Chen AAML 114 3 0 21 Dec 2024
Federated Learning and RAG Integration: A Scalable Approach for Medical Large Language Models Jincheol Jung Hongju Jeong Eui-Nam Huh 91 0 0 18 Dec 2024
Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architectures Yicheng Zhang Zhen Qin Zhaomin Wu Jian Hou Shuiguang Deng 80 2 0 28 Nov 2024
HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-tuned LLM with Quantization Yang Su Na Yan Yansha Deng Mischa Dohler Robert Schober 37 0 0 10 Nov 2024
Federated Instruction Tuning of LLMs with Domain Coverage Augmentation Zezhou Wang Yaxin Du Zhuzhong Qian Yugang Jiang Zhuzhong Qian Siheng Chen FedML 188 0 0 30 Sep 2024
On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists Dongyang Fan Bettina Messmer N. Doikov Martin Jaggi MoMe MoE 50 2 0 20 Sep 2024
On ADMM in Heterogeneous Federated Learning: Personalization, Robustness, and Fairness Shengkun Zhu Jinshan Zeng Sheng Wang Yuan Sun Xiaodong Li Yuan Yao Zhiyong Peng 52 0 0 23 Jul 2024
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 42 43 0 09 Jul 2024
FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Model Feijie Wu Zitao Li Yaliang Li Bolin Ding Jing Gao 34 42 0 25 Jun 2024
FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuning Jiaxing Qi Zhongzhi Luan Shaohan Huang Carol J. Fung Hailong Yang Depei Qian 36 13 0 12 Jun 2024
DAGER: Exact Gradient Inversion for Large Language Models Ivo Petrov Dimitar I. Dimitrov Maximilian Baader Mark Niklas Muller Martin Vechev FedML 63 3 0 24 May 2024
FL-TAC: Enhanced Fine-Tuning in Federated Learning via Low-Rank, Task-Specific Adapter Clustering Siqi Ping Yuzhu Mao Yang Liu Xiao-Ping Zhang Wenbo Ding FedML 31 4 0 23 Apr 2024
Personalized Collaborative Fine-Tuning for On-Device Large Language Models Nicolas Wagner Dongyang Fan Martin Jaggi 39 3 0 15 Apr 2024
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 42 32 0 11 Dec 2023
FedPEAT: Convergence of Federated Learning, Parameter-Efficient Fine Tuning, and Emulator Assisted Tuning for Artificial Intelligence Foundation Models with Mobile Edge Computing Terence Jie Chua Wen-li Yu Junfeng Zhao Kwok-Yan Lam FedML 32 5 0 26 Oct 2023
A Survey on Privacy in Graph Neural Networks: Attacks, Preservation, and Applications Yi Zhang Yuying Zhao Zhaoqing Li Xueqi Cheng Yu-Chiang Frank Wang Olivera Kotevska Philip S. Yu Tyler Derr 31 10 0 31 Aug 2023
Federated Few-shot Learning Song Wang Xingbo Fu Kaize Ding Chen Chen Huiyuan Chen Jundong Li FedML 49 22 0 17 Jun 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 171 588 0 06 Apr 2023
When Do Curricula Work in Federated Learning? Saeed Vahidian Sreevatsank Kadaveru Woo-Ram Baek Weijia Wang Vyacheslav Kungurtsev Chong Chen M. Shah Bill Lin FedML 40 11 0 24 Dec 2022
Rethinking Normalization Methods in Federated Learning Zhixu Du Jingwei Sun Ang Li Pin-Yu Chen Jianyi Zhang H. Li Yiran Chen FedML 29 28 0 07 Oct 2022
Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces Saeed Vahidian Mahdi Morafah Weijia Wang Vyacheslav Kungurtsev Chong Chen M. Shah Bill Lin FedML 56 45 0 21 Sep 2022
Decentralized digital twins of complex dynamical systems Omer San Suraj Pawar Adil Rasheed AI4CE 41 11 0 07 Jul 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 369 12,081 0 04 Mar 2022
A Comprehensive Survey on Federated Learning: Concept and Applications Dhurgham Hassan Mahlool Mohammed Hamzah Abed FedML 23 22 0 23 Jan 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 215 1,663 0 15 Oct 2021
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 238 814 0 14 Oct 2021
Federated Natural Language Generation for Personalized Dialogue System Yujie Lu Chao-Wei Huang Huanli Zhan Yong Zhuang FedML 85 9 0 13 Oct 2021
Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneity Saeed Vahidian Mahdi Morafah Bill Lin 64 58 0 02 May 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,872 0 18 Apr 2021
Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity Amirhossein Reisizadeh Isidoros Tziotis Hamed Hassani Aryan Mokhtari Ramtin Pedarsani FedML 172 99 0 28 Dec 2020
FedML: A Research Library and Benchmark for Federated Machine Learning Chaoyang He Songze Li Jinhyun So Xiao Zeng Mi Zhang ... Yang Liu Ramesh Raskar Qiang Yang M. Annavaram Salman Avestimehr FedML 168 566 0 27 Jul 2020
FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning Suyu Ge Fangzhao Wu Chuhan Wu Tao Qi Yongfeng Huang Xing Xie 158 57 0 20 Mar 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 299 6,984 0 20 Apr 2018