Scaling Language Model Size in Cross-Device Federated Learning

Scaling Language Model Size in Cross-Device Federated Learning

31 March 2022

Theresa Breiner

Lara McConnaughey

Papers citing "Scaling Language Model Size in Cross-Device Federated Learning"

11 / 11 papers shown

Title
No Need to Talk: Asynchronous Mixture of Language Models Anastasiia Filippova Angelos Katharopoulos David Grangier Ronan Collobert MoE 44 0 0 04 Oct 2024
Save It All: Enabling Full Parameter Tuning for Federated Large Language Models via Cycle Block Gradient Descent Lin Wang Zhichao Wang Xiaoying Tang 45 1 0 17 Jun 2024
Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security Youyang Qu Ming Liu Tianqing Zhu Longxiang Gao Shui Yu Wanlei Zhou MU FedML 65 2 0 14 Jun 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation Jie Xu Karthikeyan P. Saravanan Rogier van Dalen Haaris Mehmood David Tuckey Mete Ozay 56 6 0 10 May 2024
Only Send What You Need: Learning to Communicate Efficiently in Federated Multilingual Machine Translation Yun-Wei Chu Dong-Jun Han Christopher G. Brinton 28 4 0 15 Jan 2024
Jointly Reparametrized Multi-Layer Adaptation for Efficient and Private Tuning Umang Gupta Aram Galstyan Greg Ver Steeg 11 2 0 30 May 2023
TimelyFL: Heterogeneity-aware Asynchronous Federated Learning with Adaptive Partial Training Tuo Zhang Lei Gao Sunwoo Lee Mi Zhang Salman Avestimehr FedML 34 28 0 14 Apr 2023
Towards Sparsified Federated Neuroimaging Models via Weight Pruning Dimitris Stripelis Umang Gupta Nikhil J. Dhinagar Greg Ver Steeg Paul M. Thompson J. Ambite FedML 24 0 0 24 Aug 2022
DRIVE: One-bit Distributed Mean Estimation S. Vargaftik Ran Ben-Basat Amit Portnoy Gal Mendelson Y. Ben-Itzhak Michael Mitzenmacher OOD FedML 82 51 0 18 May 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 264 4,489 0 23 Jan 2020
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization Amirhossein Reisizadeh Aryan Mokhtari Hamed Hassani Ali Jadbabaie Ramtin Pedarsani FedML 174 763 0 28 Sep 2019