Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomness

16 June 2023

Papers citing "Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomness"

6 / 6 papers shown

Title
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 40 32 0 11 Dec 2023
How to Protect Copyright Data in Optimization of Large Language Models? T. Chu Zhao Song Chiwun Yang 40 29 0 23 Aug 2023
An Empirical Comparison of Optimizers for Quantum Machine Learning with SPSA-based Gradients Marco Wiedmann Marc Hölle Maniraman Periyasamy Nico Meyer Christian Ufrecht Daniel D. Scherer Axel Plinge Christopher Mutschler 77 18 0 27 Apr 2023
Sparse Random Networks for Communication-Efficient Federated Learning Berivan Isik Francesco Pase Deniz Gunduz Tsachy Weissman M. Zorzi FedML 70 52 0 30 Sep 2022
On the advantages of stochastic encoders Lucas Theis E. Agustsson 57 44 0 18 Feb 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 264 4,489 0 23 Jan 2020