QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding

7 October 2016

Dan Alistarh

Papers citing "QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding"

50 / 128 papers shown

Title
Paving the Way for Distributed Artificial Intelligence over the Air Guoqing Ma Shuping Dang Chuanting Zhang B. Shihada 27 3 0 24 Sep 2021
Scalable Average Consensus with Compressed Communications Taha Toghani César A. Uribe 22 7 0 14 Sep 2021
Fast Federated Edge Learning with Overlapped Communication and Computation and Channel-Aware Fair Client Scheduling M. E. Ozfatura Junlin Zhao Deniz Gündüz 34 15 0 14 Sep 2021
Bayesian AirComp with Sign-Alignment Precoding for Wireless Federated Learning Chanhoo Park Seunghoon Lee Namyoon Lee 34 5 0 14 Sep 2021
Fundamental limits of over-the-air optimization: Are analog schemes optimal? Shubham K. Jha Prathamesh Mayekar Himanshu Tyagi 29 7 0 11 Sep 2021
EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning S. Vargaftik Ran Ben-Basat Amit Portnoy Gal Mendelson Y. Ben-Itzhak Michael Mitzenmacher FedML 46 46 0 19 Aug 2021
ErrorCompensatedX: error compensation for variance reduced algorithms Hanlin Tang Yao Li Ji Liu Ming Yan 32 10 0 04 Aug 2021
Learning a Neural Diff for Speech Models J. Macoskey Grant P. Strimel Ariya Rastrow 18 2 0 03 Aug 2021
Rethinking gradient sparsification as total error minimization Atal Narayan Sahu Aritra Dutta A. Abdelmoniem Trambak Banerjee Marco Canini Panos Kalnis 50 56 0 02 Aug 2021
Dynamic Neural Network Architectural and Topological Adaptation and Related Methods -- A Survey Lorenz Kummer AI4CE 45 0 0 28 Jul 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 187 412 0 14 Jul 2021
ResIST: Layer-Wise Decomposition of ResNets for Distributed Training Chen Dun Cameron R. Wolfe C. Jermaine Anastasios Kyrillidis 29 21 0 02 Jul 2021
Secure Distributed Training at Scale Eduard A. Gorbunov Alexander Borzunov Michael Diskin Max Ryabinin FedML 26 15 0 21 Jun 2021
CD-SGD: Distributed Stochastic Gradient Descent with Compression and Delay Compensation Enda Yu Dezun Dong Yemao Xu Shuo Ouyang Xiangke Liao 16 5 0 21 Jun 2021
FedNL: Making Newton-Type Methods Applicable to Federated Learning M. Safaryan Rustem Islamov Xun Qian Peter Richtárik FedML 33 78 0 05 Jun 2021
Escaping Saddle Points with Compressed SGD Dmitrii Avdiukhin G. Yaroslavtsev 22 4 0 21 May 2021
Towards Demystifying Serverless Machine Learning Training Jiawei Jiang Shaoduo Gan Yue Liu Fanlin Wang Gustavo Alonso Ana Klimovic Ankit Singla Wentao Wu Ce Zhang 19 122 0 17 May 2021
DP-SIGNSGD: When Efficiency Meets Privacy and Robustness Lingjuan Lyu FedML AAML 27 19 0 11 May 2021
Coded Gradient Aggregation: A Tradeoff Between Communication Costs at Edge Nodes and at Helper Nodes B. Sasidharan Anoop Thomas 28 9 0 06 May 2021
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges Mingzhe Chen Deniz Gündüz Kaibin Huang Walid Saad M. Bennis Aneta Vulgarakis Feljan H. Vincent Poor 45 402 0 05 Apr 2021
Federated Learning: A Signal Processing Perspective Tomer Gafni Nir Shlezinger Kobi Cohen Yonina C. Eldar H. Vincent Poor FedML 39 129 0 31 Mar 2021
Learned Gradient Compression for Distributed Deep Learning L. Abrahamyan Yiming Chen Giannis Bekoulis Nikos Deligiannis 40 46 0 16 Mar 2021
Efficient Randomized Subspace Embeddings for Distributed Optimization under a Communication Budget R. Saha Mert Pilanci Andrea J. Goldsmith 34 5 0 13 Mar 2021
Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiency Yuyang Deng M. Mahdavi 35 59 0 25 Feb 2021
IntSGD: Adaptive Floatless Compression of Stochastic Gradients Konstantin Mishchenko Bokun Wang D. Kovalev Peter Richtárik 75 15 0 16 Feb 2021
MARINA: Faster Non-Convex Distributed Learning with Compression Eduard A. Gorbunov Konstantin Burlachenko Zhize Li Peter Richtárik 44 109 0 15 Feb 2021
Communication-efficient Distributed Cooperative Learning with Compressed Beliefs Taha Toghani César A. Uribe 27 15 0 14 Feb 2021
Sparse-Push: Communication- & Energy-Efficient Decentralized Distributed Learning over Directed & Time-Varying Graphs with non-IID Datasets Sai Aparna Aketi Amandeep Singh J. Rabaey 29 10 0 10 Feb 2021
1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed Hanlin Tang Shaoduo Gan A. A. Awan Samyam Rajbhandari Conglong Li Xiangru Lian Ji Liu Ce Zhang Yuxiong He AI4CE 45 84 0 04 Feb 2021
Federated Learning over Wireless Device-to-Device Networks: Algorithms and Convergence Analysis Hong Xing Osvaldo Simeone Suzhi Bi 50 94 0 29 Jan 2021
Time-Correlated Sparsification for Communication-Efficient Federated Learning Emre Ozfatura Kerem Ozfatura Deniz Gunduz FedML 43 47 0 21 Jan 2021
FedNS: Improving Federated Learning for collaborative image classification on mobile clients Yaoxin Zhuo Baoxin Li FedML 21 14 0 20 Jan 2021
Activation Density based Mixed-Precision Quantization for Energy Efficient Neural Networks Karina Vasquez Yeshwanth Venkatesha Abhiroop Bhattacharjee Abhishek Moitra Priyadarshini Panda MQ 48 15 0 12 Jan 2021
Anytime Minibatch with Delayed Gradients H. Al-Lawati S. Draper 25 0 0 15 Dec 2020
CatFedAvg: Optimising Communication-efficiency and Classification Accuracy in Federated Learning D. Sarkar Sumit Rai Ankur Narang FedML 26 2 0 14 Nov 2020
Coded Computing for Low-Latency Federated Learning over Wireless Edge Networks Saurav Prakash S. Dhakal M. Akdeniz Yair Yona S. Talwar Salman Avestimehr N. Himayat FedML 35 92 0 12 Nov 2020
Improving Neural Network Training in Low Dimensional Random Bases Frithjof Gressmann Zach Eaton-Rosen Carlo Luschi 30 28 0 09 Nov 2020
Optimal Client Sampling for Federated Learning Wenlin Chen Samuel Horváth Peter Richtárik FedML 47 192 0 26 Oct 2020
Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters Filip Hanzely 42 0 0 26 Aug 2020
Federated Learning for Channel Estimation in Conventional and RIS-Assisted Massive MIMO Ahmet M. Elbir Sinem Coleri 39 131 0 25 Aug 2020
Periodic Stochastic Gradient Descent with Momentum for Decentralized Training Hongchang Gao Heng-Chiao Huang 23 25 0 24 Aug 2020
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications Jihong Park S. Samarakoon Anis Elgabli Joongheon Kim M. Bennis Seong-Lyun Kim Mérouane Debbah 39 161 0 06 Aug 2020
Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge Chaoyang He M. Annavaram A. Avestimehr FedML 32 23 0 28 Jul 2020
Breaking the Communication-Privacy-Accuracy Trilemma Wei-Ning Chen Peter Kairouz Ayfer Özgür 14 116 0 22 Jul 2020
Byzantine-Resilient Secure Federated Learning Jinhyun So Başak Güler A. Avestimehr FedML 19 238 0 21 Jul 2020
Incentives for Federated Learning: a Hypothesis Elicitation Approach Yang Liu Jiaheng Wei FedML 42 21 0 21 Jul 2020
Deep Anomaly Detection for Time-series Data in Industrial IoT: A Communication-Efficient On-device Federated Learning Approach Yi Liu S. Garg Jiangtian Nie Yan Zhang Zehui Xiong Jiawen Kang M. S. Hossain FedML 39 378 0 19 Jul 2020
DS-Sync: Addressing Network Bottlenecks with Divide-and-Shuffle Synchronization for Distributed DNN Training Weiyan Wang Cengguang Zhang Liu Yang Kai Chen Kun Tan 31 12 0 07 Jul 2020
Federated Learning with Compression: Unified Analysis and Sharp Guarantees Farzin Haddadpour Mohammad Mahdi Kamani Aryan Mokhtari M. Mahdavi FedML 42 274 0 02 Jul 2020
Is Network the Bottleneck of Distributed Training? Zhen Zhang Chaokun Chang Yanghua Peng Yida Wang R. Arora Xin Jin 25 70 0 17 Jun 2020