Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 616 papers shown

Title
On the Utility of Gradient Compression in Distributed Training Systems Saurabh Agarwal Hongyi Wang Shivaram Venkataraman Dimitris Papailiopoulos 41 46 0 28 Feb 2021
Constrained Differentially Private Federated Learning for Low-bandwidth Devices Raouf Kerkouche G. Ács C. Castelluccia P. Genevès 29 7 0 27 Feb 2021
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout Samuel Horváth Stefanos Laskaridis Mario Almeida Ilias Leondiadis Stylianos I. Venieris Nicholas D. Lane 189 268 0 26 Feb 2021
Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiency Yuyang Deng M. Mahdavi 35 59 0 25 Feb 2021
Peering Beyond the Gradient Veil with Distributed Auto Differentiation Bradley T. Baker Aashis Khanal Vince D. Calhoun Barak A. Pearlmutter Sergey Plis 23 1 0 18 Feb 2021
Data-Aware Device Scheduling for Federated Edge Learning Afaf Taik Zoubeir Mlika Soumaya Cherkaoui 17 38 0 18 Feb 2021
DESED-FL and URBAN-FL: Federated Learning Datasets for Sound Event Detection David S. Johnson Wolfgang Lorenz Michael Taenzer S. I. Mimilakis S. Grollmisch J. Abeßer Hanna M. Lukashevich FedML 35 13 0 17 Feb 2021
An Information-Theoretic Justification for Model Pruning Berivan Isik Tsachy Weissman Albert No 95 35 0 16 Feb 2021
Task-oriented Communication Design in Cyber-Physical Systems: A Survey on Theory and Applications Arsham Mostaani T. Vu Shree Krishna Sharma Van-Dinh Nguyen Qi Liao Symeon Chatzinotas 27 16 0 14 Feb 2021
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients A. Mitra Rayana H. Jaafar George J. Pappas Hamed Hassani FedML 55 157 0 14 Feb 2021
Distribution Adaptive INT8 Quantization for Training CNNs Kang Zhao Sida Huang Pan Pan Yinghan Li Yingya Zhang Zhenyu Gu Yinghui Xu MQ 30 63 0 09 Feb 2021
Large-Scale Training System for 100-Million Classification at Alibaba Liuyihan Song Pan Pan Kang Zhao Hao Yang Yiming Chen Yingya Zhang Yinghui Xu Rong Jin 40 23 0 09 Feb 2021
DeepReduce: A Sparse-tensor Communication Framework for Distributed Deep Learning Kelly Kostopoulou Hang Xu Aritra Dutta Xin Li A. Ntoulas Panos Kalnis 24 7 0 05 Feb 2021
Federated mmWave Beam Selection Utilizing LIDAR Data Mahdi Boloursaz Mashhadi Mikolaj Jankowski Tze-Yang Tung S. Kobus Deniz Gunduz 14 56 0 04 Feb 2021
Horizontally Fused Training Array: An Effective Hardware Utilization Squeezer for Training Novel Deep Learning Models Shang Wang Peiming Yang Yuxuan Zheng Xuelong Li Gennady Pekhimenko 16 22 0 03 Feb 2021
FEDZIP: A Compression Framework for Communication-Efficient Federated Learning Amirhossein Malekijoo Mohammad Javad Fadaeieslam Hanieh Malekijou Morteza Homayounfar F. Alizadeh-Shabdiz Reza Rawassizadeh FedML 34 54 0 02 Feb 2021
An Efficient Statistical-based Gradient Compression Technique for Distributed Training Systems A. Abdelmoniem Ahmed Elzanaty Mohamed-Slim Alouini Marco Canini 63 75 0 26 Jan 2021
Time-Correlated Sparsification for Communication-Efficient Federated Learning Emre Ozfatura Kerem Ozfatura Deniz Gunduz FedML 43 47 0 21 Jan 2021
Sum-Rate-Distortion Function for Indirect Multiterminal Source Coding in Federated Learning Naifu Zhang M. Tao Jia Wang FedML 24 4 0 21 Jan 2021
DynaComm: Accelerating Distributed CNN Training between Edges and Clouds through Dynamic Communication Scheduling Shangming Cai Dongsheng Wang Haixia Wang Yongqiang Lyu Guangquan Xu Xi Zheng A. Vasilakos 31 6 0 20 Jan 2021
Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networks Jie Xu Heqiang Wang Lixing Chen FedML 58 43 0 10 Jan 2021
FLAME: Taming Backdoors in Federated Learning (Extended Version 1) T. D. Nguyen Phillip Rieger Huili Chen Hossein Yalame Helen Mollering ... Azalia Mirhoseini S. Zeitouni F. Koushanfar A. Sadeghi T. Schneider AAML 32 26 0 06 Jan 2021
CADA: Communication-Adaptive Distributed Adam Tianyi Chen Ziye Guo Yuejiao Sun W. Yin ODL 14 24 0 31 Dec 2020
CosSGD: Communication-Efficient Federated Learning with a Simple Cosine-Based Quantization Yang He Hui-Po Wang M. Zenk Mario Fritz FedML MQ 27 8 0 15 Dec 2020
Quantizing data for distributed learning Osama A. Hanna Yahya H. Ezzeldin Christina Fragouli Suhas Diggavi FedML 44 20 0 14 Dec 2020
FedHome: Cloud-Edge based Personalized Federated Learning for In-Home Health Monitoring Qiong Wu Xu Chen Zhi Zhou Junshan Zhang FedML 161 272 0 14 Dec 2020
Communication-Efficient Federated Learning with Compensated Overlap-FedAvg Yuhao Zhou Qing Ye Jiancheng Lv FedML 26 122 0 12 Dec 2020
Distributed Training of Graph Convolutional Networks using Subgraph Approximation Alexandra Angerd Keshav Balasubramanian M. Annavaram GNN 31 8 0 09 Dec 2020
Towards Communication-efficient and Attack-Resistant Federated Edge Learning for Industrial Internet of Things Yi Liu Ruihui Zhao Jiawen Kang A. Yassine Dusit Niyato Jia-Jie Peng FedML 75 35 0 08 Dec 2020
Faster Non-Convex Federated Learning via Global and Local Momentum Rudrajit Das Anish Acharya Abolfazl Hashemi Sujay Sanghavi Inderjit S. Dhillon Ufuk Topcu FedML 40 82 0 07 Dec 2020
Distributed Training and Optimization Of Neural Networks J. Vlimant Junqi Yin AI4CE 6 2 0 03 Dec 2020
Robust Federated Learning with Noisy Labels Seunghan Yang Hyoungseob Park Junyoung Byun Changick Kim FedML NoLa 24 77 0 03 Dec 2020
Edge-assisted Democratized Learning Towards Federated Analytics Shashi Raj Pandey Minh N. H. Nguyen Tri Nguyen Dang N. H. Tran K. Thar Zhu Han Choong Seon Hong FedML 20 22 0 01 Dec 2020
Gradient Sparsification Can Improve Performance of Differentially-Private Convex Machine Learning F. Farokhi 33 4 0 30 Nov 2020
Federated learning with class imbalance reduction Miao Yang Akitanoshou Wong Hongbin Zhu Haifeng Wang H. Qian FedML 14 121 0 23 Nov 2020
A Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learning Xinyi Xu Lingjuan Lyu FedML 31 69 0 20 Nov 2020
EasyTransfer -- A Simple and Scalable Deep Transfer Learning Platform for NLP Applications Minghui Qiu Peng Li Chengyu Wang Hanjie Pan Yaliang Li ... Jun Yang Yaliang Li Jun Huang Deng Cai Wei Lin VLM SyDa 39 20 0 18 Nov 2020
Contrastive Weight Regularization for Large Minibatch SGD Qiwei Yuan Weizhe Hua Yi Zhou Cunxi Yu OffRL 27 1 0 17 Nov 2020
An Exploratory Analysis on Users' Contributions in Federated Learning Jiyue Huang Rania Talbi Zilong Zhao S. Bouchenak L. Chen Stefanie Roos FedML 26 30 0 13 Nov 2020
Distributed Sparse SGD with Majority Voting Kerem Ozfatura Emre Ozfatura Deniz Gunduz FedML 46 4 0 12 Nov 2020
Compression Boosts Differentially Private Federated Learning Raouf Kerkouche G. Ács C. Castelluccia P. Genevès FedML 30 29 0 10 Nov 2020
Improving Neural Network Training in Low Dimensional Random Bases Frithjof Gressmann Zach Eaton-Rosen Carlo Luschi 30 28 0 09 Nov 2020
Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning Nader Bouacida Jiahui Hou H. Zang Xin Liu FedML 25 75 0 08 Nov 2020
FederBoost: Private Federated Learning for GBDT Zhihua Tian Rui Zhang Xiaoyang Hou Jian-wei Liu K. Ren Jian Liu Kui Ren FedML AI4CE 47 66 0 05 Nov 2020
Local SGD: Unified Theory and New Efficient Methods Eduard A. Gorbunov Filip Hanzely Peter Richtárik FedML 37 109 0 03 Nov 2020
Accordion: Adaptive Gradient Communication via Critical Learning Regime Identification Saurabh Agarwal Hongyi Wang Kangwook Lee Shivaram Venkataraman Dimitris Papailiopoulos 34 25 0 29 Oct 2020
Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusters Shaoshuai Shi Xianhao Zhou Shutao Song Xingyao Wang Zilin Zhu ... Chenyang Guo Bo Yang Zhibo Chen Yongjian Wu Xiaowen Chu GNN 23 55 0 20 Oct 2020
FPRaker: A Processing Element For Accelerating Neural Network Training Omar Mohamed Awad Mostafa Mahmoud Isak Edo Vivancos Ali Hadi Zadeh Ciaran Bannon Anand Jayarajan Gennady Pekhimenko Andreas Moshovos 28 15 0 15 Oct 2020
Federated Learning in Adversarial Settings Raouf Kerkouche G. Ács C. Castelluccia FedML 21 15 0 15 Oct 2020
Fairness-aware Agnostic Federated Learning Wei Du Depeng Xu Xintao Wu Hanghang Tong FedML 18 128 0 10 Oct 2020