v1v2v3 (latest)

Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

ArXiv (abs)PDF HTML Github (222★)

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 625 papers shown

Title
MergeComp: A Compression Scheduler for Scalable Communication-Efficient Distributed Training Zhuang Wang X. Wu T. Ng GNN 32 4 0 28 Mar 2021
Learned Gradient Compression for Distributed Deep Learning L. Abrahamyan Yiming Chen Giannis Bekoulis Nikos Deligiannis 106 46 0 16 Mar 2021
EventGraD: Event-Triggered Communication in Parallel Machine Learning Soumyadip Ghosh B. Aquino V. Gupta FedML 101 9 0 12 Mar 2021
Pufferfish: Communication-efficient Models At No Extra Cost Hongyi Wang Saurabh Agarwal Dimitris Papailiopoulos 85 59 0 05 Mar 2021
Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture S. Min Kun Wu Sitao Huang Mert Hidayetouglu Jinjun Xiong Eiman Ebrahimi Deming Chen Wen-mei W. Hwu GNN 102 69 0 04 Mar 2021
Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices Max Ryabinin Eduard A. Gorbunov Vsevolod Plokhotnyuk Gennady Pekhimenko 133 36 0 04 Mar 2021
Temporal-Structure-Assisted Gradient Aggregation for Over-the-Air Federated Edge Learning Dian Fan Xiaojun Yuan Y. Zhang FedML 103 24 0 03 Mar 2021
Adaptive Transmission Scheduling in Wireless Networks for Asynchronous Federated Learning Hyun-Suk Lee Jang-Won Lee 139 55 0 02 Mar 2021
Towards Personalized Federated Learning A. Tan Han Yu Li-zhen Cui Qiang Yang FedML AI4CE 339 887 0 01 Mar 2021
On the Utility of Gradient Compression in Distributed Training Systems Saurabh Agarwal Hongyi Wang Shivaram Venkataraman Dimitris Papailiopoulos 107 47 0 28 Feb 2021
Constrained Differentially Private Federated Learning for Low-bandwidth Devices Raouf Kerkouche G. Ács C. Castelluccia P. Genevès 60 7 0 27 Feb 2021
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout Samuel Horváth Stefanos Laskaridis Mario Almeida Ilias Leondiadis Stylianos I. Venieris Nicholas D. Lane 304 275 0 26 Feb 2021
Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiency Yuyang Deng M. Mahdavi 107 61 0 25 Feb 2021
Peering Beyond the Gradient Veil with Distributed Auto Differentiation Bradley T. Baker Aashis Khanal Vince D. Calhoun Barak A. Pearlmutter Sergey Plis 47 1 0 18 Feb 2021
Data-Aware Device Scheduling for Federated Edge Learning Afaf Taik Zoubeir Mlika Soumaya Cherkaoui 66 39 0 18 Feb 2021
DESED-FL and URBAN-FL: Federated Learning Datasets for Sound Event Detection David S. Johnson Wolfgang Lorenz Michael Taenzer S. I. Mimilakis S. Grollmisch J. Abeßer Hanna M. Lukashevich FedML 67 15 0 17 Feb 2021
An Information-Theoretic Justification for Model Pruning Berivan Isik Tsachy Weissman Albert No 164 37 0 16 Feb 2021
Task-oriented Communication Design in Cyber-Physical Systems: A Survey on Theory and Applications Arsham Mostaani T. Vu Shree Krishna Sharma Van-Dinh Nguyen Qi Liao Symeon Chatzinotas 82 17 0 14 Feb 2021
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients A. Mitra Rayana H. Jaafar George J. Pappas Hamed Hassani FedML 114 160 0 14 Feb 2021
Distribution Adaptive INT8 Quantization for Training CNNs Kang Zhao Sida Huang Pan Pan Yinghan Li Yingya Zhang Zhenyu Gu Yinghui Xu MQ 114 68 0 09 Feb 2021
Large-Scale Training System for 100-Million Classification at Alibaba Liuyihan Song Pan Pan Kang Zhao Hao Yang Yiming Chen Yingya Zhang Yinghui Xu Rong Jin 86 24 0 09 Feb 2021
DeepReduce: A Sparse-tensor Communication Framework for Distributed Deep Learning Kelly Kostopoulou Hang Xu Aritra Dutta Xin Li A. Ntoulas Panos Kalnis 43 7 0 05 Feb 2021
Federated mmWave Beam Selection Utilizing LIDAR Data Mahdi Boloursaz Mashhadi Mikolaj Jankowski Tze-Yang Tung S. Kobus Deniz Gunduz 41 56 0 04 Feb 2021
Horizontally Fused Training Array: An Effective Hardware Utilization Squeezer for Training Novel Deep Learning Models Shang Wang Peiming Yang Yuxuan Zheng Xuelong Li Gennady Pekhimenko 82 22 0 03 Feb 2021
FEDZIP: A Compression Framework for Communication-Efficient Federated Learning Amirhossein Malekijoo Mohammad Javad Fadaeieslam Hanieh Malekijou Morteza Homayounfar F. Alizadeh-Shabdiz Reza Rawassizadeh FedML 94 55 0 02 Feb 2021
An Efficient Statistical-based Gradient Compression Technique for Distributed Training Systems A. Abdelmoniem Ahmed Elzanaty Mohamed-Slim Alouini Marco Canini 132 77 0 26 Jan 2021
Time-Correlated Sparsification for Communication-Efficient Federated Learning Emre Ozfatura Kerem Ozfatura Deniz Gunduz FedML 89 49 0 21 Jan 2021
Sum-Rate-Distortion Function for Indirect Multiterminal Source Coding in Federated Learning Naifu Zhang M. Tao Jia Wang FedML 58 4 0 21 Jan 2021
DynaComm: Accelerating Distributed CNN Training between Edges and Clouds through Dynamic Communication Scheduling Shangming Cai Dongsheng Wang Haixia Wang Yongqiang Lyu Guangquan Xu Xi Zheng A. Vasilakos 66 6 0 20 Jan 2021
Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networks Jie Xu Heqiang Wang Lixing Chen FedML 125 45 0 10 Jan 2021
FLAME: Taming Backdoors in Federated Learning (Extended Version 1) T. D. Nguyen Phillip Rieger Huili Chen Hossein Yalame Helen Mollering ... Azalia Mirhoseini S. Zeitouni F. Koushanfar A. Sadeghi T. Schneider AAML 89 25 0 06 Jan 2021
CADA: Communication-Adaptive Distributed Adam Tianyi Chen Ziye Guo Yuejiao Sun W. Yin ODL 49 24 0 31 Dec 2020
CosSGD: Communication-Efficient Federated Learning with a Simple Cosine-Based Quantization Yang He Hui-Po Wang M. Zenk Mario Fritz FedML MQ 82 8 0 15 Dec 2020
Quantizing data for distributed learning Osama A. Hanna Yahya H. Ezzeldin Christina Fragouli Suhas Diggavi FedML 108 21 0 14 Dec 2020
FedHome: Cloud-Edge based Personalized Federated Learning for In-Home Health Monitoring Qiong Wu Xu Chen Zhi Zhou Junshan Zhang FedML 239 287 0 14 Dec 2020
Communication-Efficient Federated Learning with Compensated Overlap-FedAvg Yuhao Zhou Qing Ye Jiancheng Lv FedML 61 129 0 12 Dec 2020
Distributed Training of Graph Convolutional Networks using Subgraph Approximation Alexandra Angerd Keshav Balasubramanian M. Annavaram GNN 54 8 0 09 Dec 2020
Towards Communication-efficient and Attack-Resistant Federated Edge Learning for Industrial Internet of Things Yi Liu Ruihui Zhao Jiawen Kang A. Yassine Dusit Niyato Jia-Jie Peng FedML 120 35 0 08 Dec 2020
Faster Non-Convex Federated Learning via Global and Local Momentum Rudrajit Das Anish Acharya Abolfazl Hashemi Sujay Sanghavi Inderjit S. Dhillon Ufuk Topcu FedML 100 84 0 07 Dec 2020
Distributed Training and Optimization Of Neural Networks J. Vlimant Junqi Yin AI4CE 30 2 0 03 Dec 2020
Robust Federated Learning with Noisy Labels Seunghan Yang Hyoungseob Park Junyoung Byun Changick Kim FedML NoLa 64 80 0 03 Dec 2020
Edge-assisted Democratized Learning Towards Federated Analytics Shashi Raj Pandey Minh N. H. Nguyen Tri Nguyen Dang N. H. Tran K. Thar Zhu Han Choong Seon Hong FedML 44 22 0 01 Dec 2020
Gradient Sparsification Can Improve Performance of Differentially-Private Convex Machine Learning F. Farokhi 66 5 0 30 Nov 2020
Federated learning with class imbalance reduction Miao Yang Akitanoshou Wong Hongbin Zhu Haifeng Wang H. Qian FedML 69 125 0 23 Nov 2020
A Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learning Xinyi Xu Lingjuan Lyu FedML 125 71 0 20 Nov 2020
EasyTransfer -- A Simple and Scalable Deep Transfer Learning Platform for NLP Applications Minghui Qiu Peng Li Chengyu Wang Hanjie Pan Yaliang Li ... Jun Yang Yaliang Li Jun Huang Deng Cai Wei Lin VLM SyDa 109 20 0 18 Nov 2020
Contrastive Weight Regularization for Large Minibatch SGD Qiwei Yuan Weizhe Hua Yi Zhou Cunxi Yu OffRL 83 1 0 17 Nov 2020
An Exploratory Analysis on Users' Contributions in Federated Learning Jiyue Huang Rania Talbi Zilong Zhao S. Bouchenak L. Chen Stefanie Roos FedML 85 31 0 13 Nov 2020
Distributed Sparse SGD with Majority Voting Kerem Ozfatura Emre Ozfatura Deniz Gunduz FedML 76 4 0 12 Nov 2020
Compression Boosts Differentially Private Federated Learning Raouf Kerkouche G. Ács C. Castelluccia P. Genevès FedML 75 29 0 10 Nov 2020