Communication Compression for Decentralized Training

17 March 2018

Tong Zhang

Papers citing "Communication Compression for Decentralized Training"

28 / 28 papers shown

Title
Fully First-Order Methods for Decentralized Bilevel Optimization Xiaoyu Wang Xuxing Chen Shiqian Ma Tong Zhang 86 0 0 25 Oct 2024
Ordered Momentum for Asynchronous SGD Chang-Wei Shi Yi-Rui Yang Wu-Jun Li ODL 123 0 0 27 Jul 2024
SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data Sakshi Choudhary Sai Aparna Aketi Kaushik Roy FedML 80 0 0 22 May 2024
Faster Convergence with Less Communication: Broadcast-Based Subgraph Sampling for Decentralized Learning over Wireless Networks Daniel Pérez Herrera Zheng Chen Erik G. Larsson 102 1 0 24 Jan 2024
Sparsified SGD with Memory Sebastian U. Stich Jean-Baptiste Cordonnier Martin Jaggi 71 749 0 20 Sep 2018
Don't Use Large Mini-Batches, Use Local SGD Tao R. Lin Sebastian U. Stich Kumar Kshitij Patel Martin Jaggi 111 433 0 22 Aug 2018
COLA: Decentralized Linear Learning Lie He An Bian Martin Jaggi 77 120 0 13 Aug 2018
Local SGD Converges Fast and Communicates Little Sebastian U. Stich FedML 164 1,061 0 24 May 2018
Training DNNs with Hybrid Block Floating Point M. Drumond Tao R. Lin Martin Jaggi Babak Falsafi 45 96 0 04 Apr 2018
Gradient Sparsification for Communication-Efficient Distributed Optimization Jianqiao Wangni Jialei Wang Ji Liu Tong Zhang 72 525 0 26 Oct 2017
Asynchronous Decentralized Parallel Stochastic Gradient Descent Xiangru Lian Wei Zhang Ce Zhang Ji Liu ODL 42 501 0 18 Oct 2017
Fully Decentralized Policies for Multi-Agent Systems: An Information Theoretic Approach Roel Dobbe David Fridovich-Keil Claire Tomlin 40 42 0 20 Jul 2017
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent Xiangru Lian Ce Zhang Huan Zhang Cho-Jui Hsieh Wei Zhang Ji Liu 48 1,224 0 25 May 2017
A decentralized proximal-gradient method with network independent step-sizes and separated convergence rates Zhi Li W. Shi Ming Yan 49 226 0 25 Apr 2017
Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning El-Mahdi El-Mhamdi R. Guerraoui Hadrien Hendrikx Alexandre Maurer 38 28 0 10 Apr 2017
Communication-Efficient Algorithms for Decentralized and Stochastic Optimization Guanghui Lan Soomin Lee Yi Zhou 71 218 0 14 Jan 2017
Randomized Distributed Mean Estimation: Accuracy vs Communication Jakub Konecný Peter Richtárik FedML 115 102 0 22 Nov 2016
Distributed Mean Estimation with Limited Communication A. Suresh Felix X. Yu Sanjiv Kumar H. B. McMahan FedML 97 363 0 02 Nov 2016
Asynchronous Stochastic Gradient Descent with Delay Compensation Shuxin Zheng Qi Meng Taifeng Wang Wei Chen Nenghai Yu Zhiming Ma Tie-Yan Liu 96 314 0 27 Sep 2016
Gossip Dual Averaging for Decentralized Optimization of Pairwise Functions Igor Colin A. Bellet Joseph Salmon Stephan Clémençon FedML 63 106 0 08 Jun 2016
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 394 18,334 0 27 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.8K 193,426 0 10 Dec 2015
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems Tianqi Chen Mu Li Yutian Li Min Lin Naiyan Wang Minjie Wang Tianjun Xiao Bing Xu Chiyuan Zhang Zheng Zhang 172 2,244 0 03 Dec 2015
Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization Xiangru Lian Yijun Huang Y. Li Ji Liu 135 499 0 27 Jun 2015
Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming Saeed Ghadimi Guanghui Lan ODL 120 1,547 0 22 Sep 2013
HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent Feng Niu Benjamin Recht Christopher Ré Stephen J. Wright 165 2,273 0 28 Jun 2011
Distributed Delayed Stochastic Optimization Alekh Agarwal John C. Duchi 123 626 0 28 Apr 2011
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 255 685 0 07 Dec 2010