Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

5 December 2017

Song Han

Papers citing "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training"

50 / 616 papers shown

Title
Efficient Strong Scaling Through Burst Parallel Training S. Park Joshua Fried Sunghyun Kim Mohammad Alizadeh Adam Belay GNN LRM 32 10 0 19 Dec 2021
Efficient Differentially Private Secure Aggregation for Federated Learning via Hardness of Learning with Errors Timothy Stevens Christian Skalka C. Vincent J. Ring Samuel Clark Joseph P. Near FedML 32 71 0 13 Dec 2021
Optimal Rate Adaption in Federated Learning with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou Jiangchuan Liu FedML 42 40 0 13 Dec 2021
SparseFed: Mitigating Model Poisoning Attacks in Federated Learning with Sparsification Ashwinee Panda Saeed Mahloujifar A. Bhagoji Supriyo Chakraborty Prateek Mittal FedML AAML 17 85 0 12 Dec 2021
Batch Label Inference and Replacement Attacks in Black-Boxed Vertical Federated Learning Yang Liu Tianyuan Zou Yan Kang Wenhan Liu Yuanqin He Zhi-qian Yi Qian Yang FedML AAML 19 19 0 10 Dec 2021
PMFL: Partial Meta-Federated Learning for heterogeneous tasks and its applications on real-world medical records Tianyi Zhang Shirui Zhang Ziwei Chen Dianbo Liu FedML 28 4 0 10 Dec 2021
FastSGD: A Fast Compressed SGD Framework for Distributed Machine Learning Keyu Yang Lu Chen Zhihao Zeng Yunjun Gao 28 9 0 08 Dec 2021
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient Xiaoge Deng Dongsheng Li Tao Sun Xicheng Lu FedML 26 0 0 08 Dec 2021
Collaborative Learning over Wireless Networks: An Introductory Overview Emre Ozfatura Deniz Gunduz H. Vincent Poor 30 11 0 07 Dec 2021
Intrinisic Gradient Compression for Federated Learning Luke Melas-Kyriazi Franklyn Wang FedML 23 3 0 05 Dec 2021
Communication-Efficient Federated Learning via Quantized Compressed Sensing Yong-Nam Oh Namyoon Lee Yo-Seb Jeon H. Vincent Poor FedML MQ 30 34 0 30 Nov 2021
Doing More by Doing Less: How Structured Partial Backpropagation Improves Deep Learning Clusters Adarsh Kumar Kausik Subramanian Shivaram Venkataraman Aditya Akella 9 4 0 20 Nov 2021
An Expectation-Maximization Perspective on Federated Learning Christos Louizos M. Reisser Joseph B. Soriaga Max Welling FedML 33 11 0 19 Nov 2021
COMET: A Novel Memory-Efficient Deep Learning Training Framework by Using Error-Bounded Lossy Compression Sian Jin Chengming Zhang Xintong Jiang Yunhe Feng Hui Guan Guanpeng Li Shuaiwen Leon Song Dingwen Tao 30 23 0 18 Nov 2021
CGX: Adaptive System Support for Communication-Efficient Deep Learning I. Markov Hamidreza Ramezanikebrya Dan Alistarh GNN 19 4 0 16 Nov 2021
CoReS: Compatible Representations via Stationarity Niccoló Biondi F. Pernici Matteo Bruni A. Bimbo OOD 29 9 0 15 Nov 2021
Edge-Native Intelligence for 6G Communications Driven by Federated Learning: A Survey of Trends and Challenges Mohammad M. Al-Quraan Lina S. Mohjazi Lina Bariah A. Centeno A. Zoha Sami Muhaidat Mérouane Debbah Muhammad Ali Imran 22 62 0 14 Nov 2021
SEOFP-NET: Compression and Acceleration of Deep Neural Networks for Speech Enhancement Using Sign-Exponent-Only Floating-Points Yu-Chen Lin Cheng Yu Y. Hsu Szu-Wei Fu Yu Tsao Tei-Wei Kuo 19 6 0 08 Nov 2021
Finite-Time Consensus Learning for Decentralized Optimization with Nonlinear Gossiping Junya Chen Sijia Wang Lawrence Carin Chenyang Tao 8 3 0 04 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 32 14 0 01 Nov 2021
Gradient Inversion with Generative Image Prior Jinwoo Jeon Jaechang Kim Kangwook Lee Sewoong Oh Jungseul Ok 30 148 0 28 Oct 2021
Federated Learning over Wireless IoT Networks with Optimized Communication and Resources Student Member Ieee Hao Chen Shaocheng Huang Deyou Zhang Ming Xiao Fellow Ieee Mikael Skoglund L. F. I. H. Vincent Poor 45 94 0 22 Oct 2021
BEV-SGD: Best Effort Voting SGD for Analog Aggregation Based Federated Learning against Byzantine Attackers Xin-Yue Fan Yue Wang Yan Huo Zhi Tian FedML 27 23 0 18 Oct 2021
EmbRace: Accelerating Sparse Communication for Distributed Training of NLP Neural Networks Shengwei Li Zhiquan Lai Dongsheng Li Yiming Zhang Xiangyu Ye Yabo Duan FedML 15 2 0 18 Oct 2021
Federated learning and next generation wireless communications: A survey on bidirectional relationship Debaditya Shome Omer Waqar Wali Ullah Khan 46 31 0 14 Oct 2021
ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training Hui-Po Wang Sebastian U. Stich Yang He Mario Fritz FedML AI4CE 36 48 0 11 Oct 2021
An Empirical Study on Compressed Decentralized Stochastic Gradient Algorithms with Overparameterized Models A. Rao Hoi-To Wai 24 0 0 09 Oct 2021
Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees Aleksandr Beznosikov Peter Richtárik Michael Diskin Max Ryabinin Alexander Gasnikov FedML 25 21 0 07 Oct 2021
S2 Reducer: High-Performance Sparse Communication to Accelerate Distributed Deep Learning Ke-shi Ge Yongquan Fu Zhiquan Lai Xiaoge Deng Dongsheng Li 31 2 0 05 Oct 2021
Solon: Communication-efficient Byzantine-resilient Distributed Training via Redundant Gradients Lingjiao Chen Leshang Chen Hongyi Wang S. Davidson Yan Sun FedML 37 1 0 04 Oct 2021
Algorithm Fairness in AI for Medicine and Healthcare Richard J. Chen Tiffany Y. Chen Jana Lipkova Judy J. Wang Drew F. K. Williamson Ming Y. Lu S. Sahai Faisal Mahmood FaML 73 45 0 01 Oct 2021
Unbiased Single-scale and Multi-scale Quantizers for Distributed Optimization S. Vineeth MQ 23 0 0 26 Sep 2021
Toward Communication Efficient Adaptive Gradient Method Xiangyi Chen Xiaoyun Li P. Li FedML 42 41 0 10 Sep 2021
On the Convergence of Decentralized Adaptive Gradient Methods Xiangyi Chen Belhal Karimi Weijie Zhao Ping Li 23 21 0 07 Sep 2021
FLASHE: Additively Symmetric Homomorphic Encryption for Cross-Silo Federated Learning Zhifeng Jiang Wen Wang Yang Liu FedML 37 49 0 02 Sep 2021
Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions Yang Wu Dingheng Wang Xiaotong Lu Fan Yang Guoqi Li W. Dong Jianbo Shi 29 18 0 30 Aug 2021
Rate distortion comparison of a few gradient quantizers Tharindu B. Adikari MQ 6 0 0 23 Aug 2021
FedSkel: Efficient Federated Learning on Heterogeneous Systems with Skeleton Gradients Update Junyu Luo Jianlei Yang Xucheng Ye Xin Guo Weisheng Zhao FedML 28 14 0 20 Aug 2021
EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning S. Vargaftik Ran Ben-Basat Amit Portnoy Gal Mendelson Y. Ben-Itzhak Michael Mitzenmacher FedML 46 46 0 19 Aug 2021
Compressing gradients by exploiting temporal correlation in momentum-SGD Tharindu B. Adikari S. Draper 8 0 0 17 Aug 2021
FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning Nam Hyeon-Woo Moon Ye-Bin Tae-Hyun Oh FedML 16 115 0 13 Aug 2021
A Distributed SGD Algorithm with Global Sketching for Deep Learning Training Acceleration Lingfei Dai Boyu Diao Chao Li Yongjun Xu 41 5 0 13 Aug 2021
Communication Optimization in Large Scale Federated Learning using Autoencoder Compressed Weight Updates Srikanth Chandar Pravin Chandran Raghavendra Bhat Avinash Chakravarthi AI4CE 38 3 0 12 Aug 2021
PRECODE - A Generic Model Extension to Prevent Deep Gradient Leakage Daniel Scheliga Patrick Mäder M. Seeland MIACV 6 36 0 10 Aug 2021
Understanding Human Innate Immune System Dependencies using Graph Neural Networks Shagufta Henna GNN AI4CE 29 0 0 05 Aug 2021
Learning a Neural Diff for Speech Models J. Macoskey Grant P. Strimel Ariya Rastrow 18 2 0 03 Aug 2021
Rethinking gradient sparsification as total error minimization Atal Narayan Sahu Aritra Dutta A. Abdelmoniem Trambak Banerjee Marco Canini Panos Kalnis 50 56 0 02 Aug 2021
Communication-Efficient Federated Learning via Predictive Coding Kai Yue Richeng Jin Chau-Wai Wong H. Dai FedML 25 14 0 02 Aug 2021
FedLab: A Flexible Federated Learning Framework Dun Zeng Siqi Liang Xiangjing Hu Hui Wang Zenglin Xu FedML 13 107 0 24 Jul 2021
RingFed: Reducing Communication Costs in Federated Learning on Non-IID Data Guang Yang Ke Mu Chunhe Song Zhijia Yang Tierui Gong FedML 24 16 0 19 Jul 2021