FwdLLM: Efficient FedLLM using Forward Gradient

FwdLLM: Efficient FedLLM using Forward Gradient

26 August 2023

Mengwei Xu

Shangguang Wang

Papers citing "FwdLLM: Efficient FedLLM using Forward Gradient"

12 / 12 papers shown

Title
Federated Adapter on Foundation Models: An Out-Of-Distribution Approach Yiyuan Yang Guodong Long Tianyi Zhou Qinghua Lu Shanshan Ye Jing Jiang OODD 168 1 0 02 May 2025
A General Pseudonymization Framework for Cloud-Based LLMs: Replacing Privacy Information in Controlled Text Generation Shilong Hou Ruilin Shang Zi Long Xianghua Fu Yin Chen 64 0 0 24 Feb 2025
Small Language Models: Survey, Measurements, and Insights Zhenyan Lu Xiang Li Dongqi Cai Rongjie Yi Fangming Liu Xiwen Zhang Nicholas D. Lane Mengwei Xu ObjD LRM 55 36 0 24 Sep 2024
A Survey of Resource-efficient LLM and Multimodal Foundation Models Mengwei Xu Wangsong Yin Dongqi Cai Rongjie Yi Daliang Xu ... Shangguang Wang Yuanchun Li Yunxin Liu Xin Jin Xuanzhe Liu VLM 78 75 0 16 Jan 2024
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 32 32 0 11 Dec 2023
Scaling Forward Gradient With Local Losses Mengye Ren Simon Kornblith Renjie Liao Geoffrey E. Hinton 78 49 0 07 Oct 2022
BBTv2: Towards a Gradient-Free Future with Large Language Models Tianxiang Sun Zhengfu He Hong Qian Yunhua Zhou Xuanjing Huang Xipeng Qiu 105 53 0 23 May 2022
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 238 805 0 14 Oct 2021
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout Samuel Horváth Stefanos Laskaridis Mario Almeida Ilias Leondiadis Stylianos I. Venieris Nicholas D. Lane 178 267 0 26 Feb 2021
Stragglers Are Not Disaster: A Hybrid Federated Learning Algorithm with Delayed Gradients Xingyu Li Zhe Qu Bo Tang Zhuo Lu FedML 67 31 0 12 Feb 2021
Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementation Su Wang Mengyuan Lee Seyyedali Hosseinalipour Roberto Morabito M. Chiang Christopher G. Brinton FedML 77 110 0 04 Jan 2021
Adaptive Federated Learning in Resource Constrained Edge Computing Systems Shiqiang Wang Tiffany Tuor Theodoros Salonidis K. Leung C. Makaya T. He Kevin S. Chan 144 1,685 0 14 Apr 2018