QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding

7 October 2016

Dan Alistarh

Papers citing "QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding"

50 / 128 papers shown

Title
Compressed Private Aggregation for Scalable and Robust Federated Learning over Massive Networks Natalie Lang Nir Shlezinger Rafael G. L. DÓliveira S. E. Rouayheb FedML 82 4 0 01 Aug 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 36 7 0 12 May 2023
FedREP: A Byzantine-Robust, Communication-Efficient and Privacy-Preserving Framework for Federated Learning Yi-Rui Yang Kun Wang Wulu Li FedML 52 3 0 09 Mar 2023
ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compression Avetik G. Karagulyan Peter Richtárik FedML 39 6 0 08 Mar 2023
PolarAir: A Compressed Sensing Scheme for Over-the-Air Federated Learning Michail Gkagkos Krishna R. Narayanan J. Chamberland C. Georghiades 47 0 0 24 Jan 2023
Gossiped and Quantized Online Multi-Kernel Learning Tomàs Ortega Hamid Jafarkhani 40 5 0 24 Jan 2023
HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association Qiong Wu Xu Chen Ouyang Tao Zhi Zhou Xiaoxi Zhang Shusen Yang Junshan Zhang 37 44 0 16 Jan 2023
Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms Vincent Plassier Alain Durmus Eric Moulines FedML 34 6 0 31 Oct 2022
GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexity Artavazd Maranjyan M. Safaryan Peter Richtárik 39 13 0 28 Oct 2022
Communication-Efficient Adam-Type Algorithms for Distributed Data Mining Wenhan Xian Feihu Huang Heng-Chiao Huang FedML 35 0 0 14 Oct 2022
Empirical Analysis on Top-k Gradient Sparsification for Distributed Deep Learning in a Supercomputing Environment Daegun Yoon Sangyoon Oh 26 0 0 18 Sep 2022
Quantization for decentralized learning under subspace constraints Roula Nassif Stefan Vlaski Marco Carpentiero Vincenzo Matta Marc Antonini Ali H. Sayed 30 29 0 16 Sep 2022
A Fast Blockchain-based Federated Learning Framework with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou FedML 22 23 0 12 Aug 2022
FedOBD: Opportunistic Block Dropout for Efficiently Training Large-scale Neural Networks through Federated Learning Yuanyuan Chen Zichen Chen Pengcheng Wu Han Yu AI4CE 22 18 0 10 Aug 2022
Towards Efficient Communications in Federated Learning: A Contemporary Survey Zihao Zhao Yuzhu Mao Yang Liu Linqi Song Ouyang Ye Xinlei Chen Wenbo Ding FedML 66 60 0 02 Aug 2022
Content Popularity Prediction Based on Quantized Federated Bayesian Learning in Fog Radio Access Networks Yu Tao Yanxiang Jiang F. Zheng Pengcheng Zhu Dusit Niyato X. You 44 6 0 23 Jun 2022
Efficient Adaptive Federated Optimization of Federated Learning for IoT Zunming Chen Hongyan Cui Ensen Wu Yu Xi 32 0 0 23 Jun 2022
sqSGD: Locally Private and Communication Efficient Federated Learning Yan Feng Tao Xiong Ruofan Wu Lingjuan Lv Leilei Shi FedML 31 2 0 21 Jun 2022
Federated Optimization Algorithms with Random Reshuffling and Gradient Compression Abdurakhmon Sadiev Grigory Malinovsky Eduard A. Gorbunov Igor Sokolov Ahmed Khaled Konstantin Burlachenko Peter Richtárik FedML 21 21 0 14 Jun 2022
Private Federated Submodel Learning with Sparsification Sajani Vithana S. Ulukus FedML 33 10 0 31 May 2022
Communication-Efficient Distributionally Robust Decentralized Learning Matteo Zecchin Marios Kountouris David Gesbert 30 9 0 31 May 2022
Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneity Kiwan Maeng Haiyu Lu Luca Melis John Nguyen Michael G. Rabbat Carole-Jean Wu FedML 51 31 0 30 May 2022
Encoded Gradients Aggregation against Gradient Leakage in Federated Learning Dun Zeng Shiyu Liu Siqi Liang Zonghang Li Hongya Wang Irwin King Zenglin Xu FedML 34 0 0 26 May 2022
Tighter Regret Analysis and Optimization of Online Federated Learning Dohyeok Kwon Jonghwan Park Songnam Hong 35 12 0 13 May 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 29 10 0 08 May 2022
PICASSO: Unleashing the Potential of GPU-centric Training for Wide-and-deep Recommender Systems Yuanxing Zhang Langshi Chen Siran Yang Man Yuan Hui-juan Yi ... Yong Li Dingyang Zhang Wei Lin Lin Qu Bo Zheng 43 32 0 11 Apr 2022
Optimising Communication Overhead in Federated Learning Using NSGA-II José Á. Morell Z. Dahi Francisco Chicano Gabriel Luque Enrique Alba FedML 35 11 0 01 Apr 2022
Local Stochastic Factored Gradient Descent for Distributed Quantum State Tomography J. Kim Taha Toghani César A. Uribe Anastasios Kyrillidis 35 3 0 22 Mar 2022
Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methods Aleksandr Beznosikov Eduard A. Gorbunov Hugo Berard Nicolas Loizou 24 49 0 15 Feb 2022
FL_PyTorch: optimization research simulator for federated learning Konstantin Burlachenko Samuel Horváth Peter Richtárik FedML 53 18 0 07 Feb 2022
Distributed Learning With Sparsified Gradient Differences Yicheng Chen Rick S. Blum Martin Takáč Brian M. Sadler 42 15 0 05 Feb 2022
TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs Weiyang Wang Moein Khazraee Zhizhen Zhong M. Ghobadi Zhihao Jia Dheevatsa Mudigere Ying Zhang A. Kewitsch 39 85 0 01 Feb 2022
Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank? Sheikh Shams Azam Seyyedali Hosseinalipour Qiang Qiu Christopher G. Brinton FedML 36 20 0 01 Feb 2022
Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization Jake B. Perazzone Shiqiang Wang Mingyue Ji Kevin S. Chan FedML 24 73 0 19 Jan 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 36 8 0 09 Jan 2022
Optimizing the Communication-Accuracy Trade-off in Federated Learning with Rate-Distortion Theory Nicole Mitchell Johannes Ballé Zachary B. Charles Jakub Konecný FedML 19 21 0 07 Jan 2022
Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats Brian Chmiel Ron Banner Elad Hoffer Hilla Ben Yaacov Daniel Soudry MQ 33 23 0 19 Dec 2021
Optimal Rate Adaption in Federated Learning with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou Jiangchuan Liu FedML 42 40 0 13 Dec 2021
Collaborative Learning over Wireless Networks: An Introductory Overview Emre Ozfatura Deniz Gunduz H. Vincent Poor 30 11 0 07 Dec 2021
Wyner-Ziv Gradient Compression for Federated Learning Kai Liang Huiru Zhong Haoning Chen Youlong Wu FedML 29 8 0 16 Nov 2021
FedGreen: Federated Learning with Fine-Grained Gradient Compression for Green Mobile Edge Computing Peichun Li Xumin Huang Miao Pan Rong Yu 50 15 0 11 Nov 2021
Solving Multi-Arm Bandit Using a Few Bits of Communication Osama A. Hanna Lin F. Yang Christina Fragouli 29 16 0 11 Nov 2021
Basis Matters: Better Communication-Efficient Second Order Methods for Federated Learning Xun Qian Rustem Islamov M. Safaryan Peter Richtárik FedML 24 23 0 02 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 32 14 0 01 Nov 2021
Optimal Compression of Locally Differentially Private Mechanisms Abhin Shah Wei-Ning Chen Johannes Ballé Peter Kairouz Lucas Theis 35 42 0 29 Oct 2021
BEV-SGD: Best Effort Voting SGD for Analog Aggregation Based Federated Learning against Byzantine Attackers Xin-Yue Fan Yue Wang Yan Huo Zhi Tian FedML 27 23 0 18 Oct 2021
Trade-offs of Local SGD at Scale: An Empirical Study Jose Javier Gonzalez Ortiz Jonathan Frankle Michael G. Rabbat Ari S. Morcos Nicolas Ballas FedML 43 19 0 15 Oct 2021
Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation Divyansh Jhunjhunwala Ankur Mallick Advait Gadhikar S. Kadhe Gauri Joshi 24 10 0 14 Oct 2021
EF21 with Bells & Whistles: Practical Algorithmic Extensions of Modern Error Feedback Ilyas Fatkhullin Igor Sokolov Eduard A. Gorbunov Zhize Li Peter Richtárik 51 46 0 07 Oct 2021
Solon: Communication-efficient Byzantine-resilient Distributed Training via Redundant Gradients Lingjiao Chen Leshang Chen Hongyi Wang S. Davidson Yan Sun FedML 37 1 0 04 Oct 2021