v1v2v3 (latest)

Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

ArXiv (abs)PDF HTML Github (222★)

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 625 papers shown

Title
Improving Neural Network Training in Low Dimensional Random Bases Frithjof Gressmann Zach Eaton-Rosen Carlo Luschi 78 28 0 09 Nov 2020
Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning Nader Bouacida Jiahui Hou H. Zang Xin Liu FedML 93 78 0 08 Nov 2020
FederBoost: Private Federated Learning for GBDT Zhihua Tian Rui Zhang Xiaoyang Hou Jian Liu K. Ren Jian Liu Kui Ren FedML AI4CE 108 70 0 05 Nov 2020
Local SGD: Unified Theory and New Efficient Methods Eduard A. Gorbunov Filip Hanzely Peter Richtárik FedML 93 111 0 03 Nov 2020
Accordion: Adaptive Gradient Communication via Critical Learning Regime Identification Saurabh Agarwal Hongyi Wang Kangwook Lee Shivaram Venkataraman Dimitris Papailiopoulos 85 25 0 29 Oct 2020
Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusters Shaoshuai Shi Xianhao Zhou Shutao Song Xingyao Wang Zilin Zhu ... Chenyang Guo Bo Yang Zhibo Chen Yongjian Wu Xiaowen Chu GNN 81 55 0 20 Oct 2020
FPRaker: A Processing Element For Accelerating Neural Network Training Omar Mohamed Awad Mostafa Mahmoud Isak Edo Vivancos Ali Hadi Zadeh Ciaran Bannon Anand Jayarajan Gennady Pekhimenko Andreas Moshovos 89 15 0 15 Oct 2020
Federated Learning in Adversarial Settings Raouf Kerkouche G. Ács C. Castelluccia FedML 48 16 0 15 Oct 2020
Fairness-aware Agnostic Federated Learning Wei Du Depeng Xu Xintao Wu Hanghang Tong FedML 90 131 0 10 Oct 2020
Optimal Gradient Compression for Distributed and Federated Learning Alyazeed Albasyoni M. Safaryan Laurent Condat Peter Richtárik FedML 67 64 0 07 Oct 2020
Sparse Communication for Training Deep Networks Negar Foroutan Martin Jaggi FedML 69 17 0 19 Sep 2020
Low-Rank Training of Deep Neural Networks for Emerging Memory Technology Albert Gural P. Nadeau M. Tikekar B. Murmann 68 5 0 08 Sep 2020
HPSGD: Hierarchical Parallel SGD With Stale Gradients Featuring Yuhao Zhou Qing Ye Hailun Zhang Jiancheng Lv 3DH 69 0 0 06 Sep 2020
PSO-PS: Parameter Synchronization with Particle Swarm Optimization for Distributed Training of Deep Neural Networks Qing Ye Y. Han Yanan Sun Jiancheng Lv 47 3 0 06 Sep 2020
On Communication Compression for Distributed Optimization on Heterogeneous Data Sebastian U. Stich 97 23 0 04 Sep 2020
ESMFL: Efficient and Secure Models for Federated Learning Sheng Lin Chenghong Wang Hongjia Li Jieren Deng Yanzhi Wang Caiwen Ding FedML 45 5 0 03 Sep 2020
Wireless for Machine Learning Henrik Hellström J. M. B. D. Silva Mohammad Mohammadi Amiri Mingzhe Chen Viktoria Fodor H. Vincent Poor Carlo Fischione 85 18 0 31 Aug 2020
Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters Filip Hanzely 87 0 0 26 Aug 2020
Periodic Stochastic Gradient Descent with Momentum for Decentralized Training Hongchang Gao Heng-Chiao Huang 61 25 0 24 Aug 2020
Federated Learning with Communication Delay in Edge Networks F. Lin Christopher G. Brinton Nicolò Michelusi FedML 72 16 0 21 Aug 2020
Adaptive Distillation for Decentralized Learning from Heterogeneous Clients Jiaxin Ma Ryo Yonetani Z. Iqbal FedML 91 12 0 18 Aug 2020
Domain-specific Communication Optimization for Distributed DNN Training Hao Wang Jingrong Chen Xinchen Wan Han Tian Jiacheng Xia Gaoxiong Zeng Weiyan Wang Kai Chen Wei Bai Junchen Jiang AI4CE 26 16 0 16 Aug 2020
Step-Ahead Error Feedback for Distributed Training with Compressed Gradient An Xu Zhouyuan Huo Heng-Chiao Huang 64 14 0 13 Aug 2020
FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching Farzin Haddadpour Belhal Karimi Ping Li Xiaoyun Li FedML 82 33 0 11 Aug 2020
Scalable and Communication-efficient Decentralized Federated Edge Learning with Multi-blockchain Framework Jiawen Kang Zehui Xiong Chunxiao Jiang Yi Liu Song Guo Yang Zhang Dusit Niyato Cyril Leung Chunyan Miao FedML 53 42 0 10 Aug 2020
A Survey on Large-scale Machine Learning Meng Wang Weijie Fu Xiangnan He Shijie Hao Xindong Wu 84 112 0 10 Aug 2020
E-Tree Learning: A Novel Decentralized Model Learning Framework for Edge AI Lei Yang Yanyan Lu Jiannong Cao Jiaming Huang Mingjin Zhang 82 20 0 04 Aug 2020
PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learning Thijs Vogels Sai Praneeth Karimireddy Martin Jaggi FedML 74 54 0 04 Aug 2020
On the Convergence of SGD with Biased Gradients Ahmad Ajalloeian Sebastian U. Stich 84 90 0 31 Jul 2020
Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge Chaoyang He M. Annavaram A. Avestimehr FedML 107 23 0 28 Jul 2020
FedML: A Research Library and Benchmark for Federated Machine Learning Chaoyang He Songze Li Jinhyun So Xiao Zeng Mi Zhang ... Yang Liu Ramesh Raskar Qiang Yang M. Annavaram Salman Avestimehr FedML 269 579 0 27 Jul 2020
Regularized Flexible Activation Function Combinations for Deep Neural Networks Renlong Jie Junbin Gao A. Vasnev Minh-Ngoc Tran AI4CE 74 6 0 26 Jul 2020
DBS: Dynamic Batch Size For Distributed Deep Neural Network Training Qing Ye Yuhao Zhou Mingjia Shi Yanan Sun Jiancheng Lv 57 11 0 23 Jul 2020
Deep Anomaly Detection for Time-series Data in Industrial IoT: A Communication-Efficient On-device Federated Learning Approach Yi Liu S. Garg Jiangtian Nie Yan Zhang Zehui Xiong Jiawen Kang M. S. Hossain FedML 76 385 0 19 Jul 2020
FetchSGD: Communication-Efficient Federated Learning with Sketching D. Rothchild Ashwinee Panda Enayat Ullah Nikita Ivkin Ion Stoica Vladimir Braverman Joseph E. Gonzalez Raman Arora FedML 100 373 0 15 Jul 2020
Analyzing and Mitigating Data Stalls in DNN Training Jayashree Mohan Amar Phanishayee Ashish Raniwala Vijay Chidambaram 86 109 0 14 Jul 2020
Adaptive Periodic Averaging: A Practical Approach to Reducing Communication in Distributed Learning Peng Jiang G. Agrawal 54 5 0 13 Jul 2020
Backdoor attacks and defenses in feature-partitioned collaborative learning Yang Liu Zhi-qian Yi Tianjian Chen AAML FedML 72 48 0 07 Jul 2020
DS-Sync: Addressing Network Bottlenecks with Divide-and-Shuffle Synchronization for Distributed DNN Training Weiyan Wang Cengguang Zhang Liu Yang Kai Chen Kun Tan 75 12 0 07 Jul 2020
Deep Partial Updating: Towards Communication Efficient Updating for On-device Inference Zhongnan Qu Cong Liu Lothar Thiele 3DH 77 3 0 06 Jul 2020
Multi-Armed Bandit Based Client Scheduling for Federated Learning Wenchao Xia Tony Q.S. Quek Kun Guo Wanli Wen Howard H. Yang Hongbo Zhu FedML 129 224 0 05 Jul 2020
Federated Learning with Compression: Unified Analysis and Sharp Guarantees Farzin Haddadpour Mohammad Mahdi Kamani Aryan Mokhtari M. Mahdavi FedML 125 281 0 02 Jul 2020
Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights Shail Dave Riyadh Baghdadi Tony Nowatzki Sasikanth Avancha Aviral Shrivastava Baoxin Li 114 85 0 02 Jul 2020
Shuffle-Exchange Brings Faster: Reduce the Idle Time During Communication for Decentralized Neural Network Training Xiang Yang FedML 20 2 0 01 Jul 2020
DEED: A General Quantization Scheme for Communication Efficiency in Bits Tian-Chun Ye Peijun Xiao Ruoyu Sun FedML MQ 40 2 0 19 Jun 2020
A Better Alternative to Error Feedback for Communication-Efficient Distributed Learning Samuel Horváth Peter Richtárik 79 60 0 19 Jun 2020
Federated Learning With Quantized Global Model Updates M. Amiri Deniz Gunduz Sanjeev R. Kulkarni H. Vincent Poor FedML 108 133 0 18 Jun 2020
Is Network the Bottleneck of Distributed Training? Zhen Zhang Chaokun Chang Yanghua Peng Yida Wang R. Arora Xin Jin 89 71 0 17 Jun 2020
The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systems Sixu Hu Yuan N. Li Xu Liu Yue Liu Zhaomin Wu Bingsheng He FedML 120 57 0 14 Jun 2020
O(1) Communication for Distributed SGD through Two-Level Gradient Averaging Subhadeep Bhattacharya Weikuan Yu Fahim Chowdhury FedML 26 2 0 12 Jun 2020