Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 616 papers shown

Title
Optimal Gradient Compression for Distributed and Federated Learning Alyazeed Albasyoni M. Safaryan Laurent Condat Peter Richtárik FedML 16 62 0 07 Oct 2020
Sparse Communication for Training Deep Networks Negar Foroutan Martin Jaggi FedML 30 16 0 19 Sep 2020
Low-Rank Training of Deep Neural Networks for Emerging Memory Technology Albert Gural P. Nadeau M. Tikekar B. Murmann 26 5 0 08 Sep 2020
HPSGD: Hierarchical Parallel SGD With Stale Gradients Featuring Yuhao Zhou Qing Ye Hailun Zhang Jiancheng Lv 3DH 27 0 0 06 Sep 2020
PSO-PS: Parameter Synchronization with Particle Swarm Optimization for Distributed Training of Deep Neural Networks Qing Ye Y. Han Yanan Sun Jiancheng Lv 28 3 0 06 Sep 2020
On Communication Compression for Distributed Optimization on Heterogeneous Data Sebastian U. Stich 53 23 0 04 Sep 2020
ESMFL: Efficient and Secure Models for Federated Learning Sheng Lin Chenghong Wang Hongjia Li Jieren Deng Yanzhi Wang Caiwen Ding FedML 27 5 0 03 Sep 2020
Wireless for Machine Learning Henrik Hellström J. M. B. D. Silva Mohammad Mohammadi Amiri Mingzhe Chen Viktoria Fodor H. Vincent Poor Carlo Fischione 22 18 0 31 Aug 2020
Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters Filip Hanzely 42 0 0 26 Aug 2020
Periodic Stochastic Gradient Descent with Momentum for Decentralized Training Hongchang Gao Heng-Chiao Huang 23 25 0 24 Aug 2020
Federated Learning with Communication Delay in Edge Networks F. Lin Christopher G. Brinton Nicolò Michelusi FedML 26 16 0 21 Aug 2020
Adaptive Distillation for Decentralized Learning from Heterogeneous Clients Jiaxin Ma Ryo Yonetani Z. Iqbal FedML 40 12 0 18 Aug 2020
Domain-specific Communication Optimization for Distributed DNN Training Hao Wang Jingrong Chen Xinchen Wan Han Tian Jiacheng Xia Gaoxiong Zeng Weiyan Wang Kai Chen Wei Bai Junchen Jiang AI4CE 14 15 0 16 Aug 2020
Step-Ahead Error Feedback for Distributed Training with Compressed Gradient An Xu Zhouyuan Huo Heng-Chiao Huang 18 14 0 13 Aug 2020
FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching Farzin Haddadpour Belhal Karimi Ping Li Xiaoyun Li FedML 58 31 0 11 Aug 2020
Scalable and Communication-efficient Decentralized Federated Edge Learning with Multi-blockchain Framework Jiawen Kang Zehui Xiong Chunxiao Jiang Yi Liu Song Guo Yang Zhang Dusit Niyato Cyril Leung Chunyan Miao FedML 38 40 0 10 Aug 2020
A Survey on Large-scale Machine Learning Meng Wang Weijie Fu Xiangnan He Shijie Hao Xindong Wu 25 110 0 10 Aug 2020
E-Tree Learning: A Novel Decentralized Model Learning Framework for Edge AI Lei Yang Yanyan Lu Jiannong Cao Jiaming Huang Mingjin Zhang 36 20 0 04 Aug 2020
PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learning Thijs Vogels Sai Praneeth Karimireddy Martin Jaggi FedML 11 54 0 04 Aug 2020
On the Convergence of SGD with Biased Gradients Ahmad Ajalloeian Sebastian U. Stich 6 84 0 31 Jul 2020
Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge Chaoyang He M. Annavaram A. Avestimehr FedML 32 23 0 28 Jul 2020
FedML: A Research Library and Benchmark for Federated Machine Learning Chaoyang He Songze Li Jinhyun So Xiao Zeng Mi Zhang ... Yang Liu Ramesh Raskar Qiang Yang M. Annavaram Salman Avestimehr FedML 168 566 0 27 Jul 2020
Regularized Flexible Activation Function Combinations for Deep Neural Networks Renlong Jie Junbin Gao A. Vasnev Minh-Ngoc Tran AI4CE 39 6 0 26 Jul 2020
DBS: Dynamic Batch Size For Distributed Deep Neural Network Training Qing Ye Yuhao Zhou Mingjia Shi Yanan Sun Jiancheng Lv 22 11 0 23 Jul 2020
Deep Anomaly Detection for Time-series Data in Industrial IoT: A Communication-Efficient On-device Federated Learning Approach Yi Liu S. Garg Jiangtian Nie Yan Zhang Zehui Xiong Jiawen Kang M. S. Hossain FedML 39 378 0 19 Jul 2020
FetchSGD: Communication-Efficient Federated Learning with Sketching D. Rothchild Ashwinee Panda Enayat Ullah Nikita Ivkin Ion Stoica Vladimir Braverman Joseph E. Gonzalez Raman Arora FedML 33 361 0 15 Jul 2020
Analyzing and Mitigating Data Stalls in DNN Training Jayashree Mohan Amar Phanishayee Ashish Raniwala Vijay Chidambaram 36 105 0 14 Jul 2020
Adaptive Periodic Averaging: A Practical Approach to Reducing Communication in Distributed Learning Peng Jiang G. Agrawal 35 5 0 13 Jul 2020
Backdoor attacks and defenses in feature-partitioned collaborative learning Yang Liu Zhi-qian Yi Tianjian Chen AAML FedML 14 48 0 07 Jul 2020
DS-Sync: Addressing Network Bottlenecks with Divide-and-Shuffle Synchronization for Distributed DNN Training Weiyan Wang Cengguang Zhang Liu Yang Kai Chen Kun Tan 31 12 0 07 Jul 2020
Deep Partial Updating: Towards Communication Efficient Updating for On-device Inference Zhongnan Qu Cong Liu Lothar Thiele 3DH 29 3 0 06 Jul 2020
Multi-Armed Bandit Based Client Scheduling for Federated Learning Wenchao Xia Tony Q.S. Quek Kun Guo Wanli Wen Howard H. Yang Hongbo Zhu FedML 55 218 0 05 Jul 2020
Federated Learning with Compression: Unified Analysis and Sharp Guarantees Farzin Haddadpour Mohammad Mahdi Kamani Aryan Mokhtari M. Mahdavi FedML 42 274 0 02 Jul 2020
Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights Shail Dave Riyadh Baghdadi Tony Nowatzki Sasikanth Avancha Aviral Shrivastava Baoxin Li 64 82 0 02 Jul 2020
Shuffle-Exchange Brings Faster: Reduce the Idle Time During Communication for Decentralized Neural Network Training Xiang Yang FedML 18 2 0 01 Jul 2020
DEED: A General Quantization Scheme for Communication Efficiency in Bits Tian-Chun Ye Peijun Xiao Ruoyu Sun FedML MQ 36 2 0 19 Jun 2020
A Better Alternative to Error Feedback for Communication-Efficient Distributed Learning Samuel Horváth Peter Richtárik 24 61 0 19 Jun 2020
Federated Learning With Quantized Global Model Updates M. Amiri Deniz Gunduz Sanjeev R. Kulkarni H. Vincent Poor FedML 24 131 0 18 Jun 2020
Is Network the Bottleneck of Distributed Training? Zhen Zhang Chaokun Chang Yanghua Peng Yida Wang R. Arora Xin Jin 25 70 0 17 Jun 2020
The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systems Sixu Hu Yuan N. Li Xu Liu Yue Liu Zhaomin Wu Bingsheng He FedML 20 54 0 14 Jun 2020
O(1) Communication for Distributed SGD through Two-Level Gradient Averaging Subhadeep Bhattacharya Weikuan Yu Fahim Chowdhury FedML 12 2 0 12 Jun 2020
Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training Hongyu Zhu Amar Phanishayee Gennady Pekhimenko 23 50 0 05 Jun 2020
UVeQFed: Universal Vector Quantization for Federated Learning Nir Shlezinger Mingzhe Chen Yonina C. Eldar H. Vincent Poor Shuguang Cui FedML MQ 24 222 0 05 Jun 2020
Federated Learning for 6G Communications: Challenges, Methods, and Future Directions Yi Liu Xingliang Yuan Zehui Xiong Jiawen Kang Xiaofei Wang Dusit Niyato FedML AI4CE 10 279 0 04 Jun 2020
Local SGD With a Communication Overhead Depending Only on the Number of Workers Artin Spiridonoff Alexander Olshevsky I. Paschalidis FedML 27 19 0 03 Jun 2020
DaSGD: Squeezing SGD Parallelization Performance in Distributed Training Using Delayed Averaging Q. Zhou Yawen Zhang Pengcheng Li Xiaoyong Liu Jun Yang Runsheng Wang Ru Huang FedML 36 2 0 31 May 2020
rTop-k: A Statistical Estimation Approach to Distributed SGD L. P. Barnes Huseyin A. Inan Berivan Isik Ayfer Özgür 32 65 0 21 May 2020
OD-SGD: One-step Delay Stochastic Gradient Descent for Distributed Training Yemao Xu Dezun Dong Weixia Xu Xiangke Liao 6 7 0 14 May 2020
SQuARM-SGD: Communication-Efficient Momentum SGD for Decentralized Optimization Navjot Singh Deepesh Data Jemin George Suhas Diggavi 19 55 0 13 May 2020
Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging Shigang Li Tal Ben-Nun Giorgi Nadiradze Salvatore Di Girolamo Nikoli Dryden Dan Alistarh Torsten Hoefler 29 15 0 30 Apr 2020