A Distributed Synchronous SGD Algorithm with Global Top-$k$
Sparsification for Low Bandwidth Networks

v1v2 (latest)

A Distributed Synchronous SGD Algorithm with Global Top- $k$ Sparsification for Low Bandwidth Networks

14 January 2019

Qiang-qiang Wang

ArXiv (abs)PDF HTML

Papers citing "A Distributed Synchronous SGD Algorithm with Global Top-$k$ Sparsification for Low Bandwidth Networks"

15 / 65 papers shown

Title
Asynchronous Federated Learning with Reduced Number of Rounds and with Differential Privacy from Less Aggregated Gaussian Noise Marten van Dijk Nhuong V. Nguyen Toan N. Nguyen Lam M. Nguyen Quoc Tran-Dinh Phuong Ha Nguyen FedML 89 29 0 17 Jul 2020
Efficient Sparse-Dense Matrix-Matrix Multiplication on GPUs Using the Customized Sparse Storage Format Shaoshuai Shi Qiang-qiang Wang Xiaowen Chu 50 10 0 29 May 2020
A Quantitative Survey of Communication Optimizations in Distributed Deep Learning Shaoshuai Shi Zhenheng Tang Xiaowen Chu Chengjian Liu Wei Wang Bo Li GNN AI4CE 47 3 0 27 May 2020
A flexible framework for communication-efficient machine learning: from HPC to IoT Sarit Khirirat Sindri Magnússon Arda Aytekin M. Johansson 51 7 0 13 Mar 2020
Communication-Efficient Distributed Deep Learning: A Comprehensive Survey Zhenheng Tang Shaoshuai Shi Wei Wang Yue Liu Xiaowen Chu 83 49 0 10 Mar 2020
Communication Contention Aware Scheduling of Multiple Deep Learning Training Jobs Qiang-qiang Wang Shaoshuai Shi Canhui Wang Xiaowen Chu 70 13 0 24 Feb 2020
Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection Zhenheng Tang Shaoshuai Shi Xiaowen Chu FedML 62 58 0 22 Feb 2020
MDLdroid: a ChainSGD-reduce Approach to Mobile Deep Learning for Personal Mobile Sensing Yu Zhang Tao Gu Xi Zhang FedML 56 21 0 07 Feb 2020
Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach Pengchao Han Shiqiang Wang K. Leung FedML 86 182 0 14 Jan 2020
MG-WFBP: Merging Gradients Wisely for Efficient Communication in Distributed Deep Learning Shaoshuai Shi Xiaowen Chu Bo Li FedML 61 25 0 18 Dec 2019
Understanding Top-k Sparsification in Distributed Deep Learning Shaoshuai Shi Xiaowen Chu Ka Chun Cheung Simon See 233 101 0 20 Nov 2019
Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees Shaoshuai Shi Zhenheng Tang Qiang-qiang Wang Kaiyong Zhao Xiaowen Chu 65 22 0 20 Nov 2019
Model Pruning Enables Efficient Federated Learning on Edge Devices Yuang Jiang Shiqiang Wang Victor Valls Bongjun Ko Wei-Han Lee Kin K. Leung Leandros Tassiulas 160 470 0 26 Sep 2019
Accelerated Sparsified SGD with Error Feedback Tomoya Murata Taiji Suzuki 53 2 0 29 May 2019
MG-WFBP: Efficient Data Communication for Distributed Synchronous SGD Algorithms Shaoshuai Shi Xiaowen Chu Bo Li FedML 91 90 0 27 Nov 2018