Decentralized Deep Learning with Arbitrary Communication Compression

22 July 2019

Papers citing "Decentralized Deep Learning with Arbitrary Communication Compression"

50 / 61 papers shown

Title
Dyn-D $^2$ P: Dynamic Differentially Private Decentralized Learning with Provable Utility Guarantee Zehan Zhu Yan Huang Xin Wang Shouling Ji Jinming Xu 26 0 0 10 May 2025
Communication Optimization for Decentralized Learning atop Bandwidth-limited Edge Networks Tingyang Sun Tuan Nguyen Ting He 35 0 0 16 Apr 2025
Accelerated Distributed Optimization with Compression and Error Feedback Yuan Gao Anton Rodomanov Jeremy Rack Sebastian U. Stich 49 0 0 11 Mar 2025
Scalable Decentralized Learning with Teleportation Yuki Takezawa Sebastian U. Stich 61 1 0 25 Jan 2025
Fully First-Order Methods for Decentralized Bilevel Optimization Xiaoyu Wang Xuxing Chen Shiqian Ma Tong Zhang 38 0 0 25 Oct 2024
SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data Sakshi Choudhary Sai Aparna Aketi Kaushik Roy FedML 45 0 0 22 May 2024
Scale-Robust Timely Asynchronous Decentralized Learning Purbesh Mitra S. Ulukus 17 1 0 30 Apr 2024
Convergence of Decentralized Stochastic Subgradient-based Methods for Nonsmooth Nonconvex functions Siyuan Zhang Nachuan Xiao Xin Liu 61 1 0 18 Mar 2024
Averaging Rate Scheduler for Decentralized Learning on Heterogeneous Data Sai Aparna Aketi Sakshi Choudhary Kaushik Roy 32 1 0 05 Mar 2024
Communication-Efficient Federated Optimization over Semi-Decentralized Networks He Wang Yuejie Chi FedML 26 2 0 30 Nov 2023
Compressed and Sparse Models for Non-Convex Decentralized Learning Andrew Campbell Hang Liu Leah Woldemariam Anna Scaglione 20 0 0 09 Nov 2023
Stability and Generalization of the Decentralized Stochastic Gradient Descent Ascent Algorithm Miaoxi Zhu Li Shen Bo Du Dacheng Tao 23 6 0 31 Oct 2023
Federated Multi-Level Optimization over Decentralized Networks Shuoguang Yang Xuezhou Zhang Mengdi Wang AI4CE 28 0 0 10 Oct 2023
Achieving Linear Speedup in Decentralized Stochastic Compositional Minimax Optimization Hongchang Gao 34 1 0 25 Jul 2023
Get More for Less in Decentralized Learning Systems Akash Dhasade Anne-Marie Kermarrec Rafael Pires Rishi Sharma Milos Vujasinovic Jeffrey Wigger 26 7 0 07 Jun 2023
Error Feedback Shines when Features are Rare Peter Richtárik Elnur Gasanov Konstantin Burlachenko 28 2 0 24 May 2023
Beyond Exponential Graph: Communication-Efficient Topologies for Decentralized Learning via Finite-time Convergence Yuki Takezawa Ryoma Sato Han Bao Kenta Niwa M. Yamada 31 9 0 19 May 2023
Homogenizing Non-IID datasets via In-Distribution Knowledge Distillation for Decentralized Learning Deepak Ravikumar Gobinda Saha Sai Aparna Aketi Kaushik Roy 18 1 0 09 Apr 2023
CoDeC: Communication-Efficient Decentralized Continual Learning Sakshi Choudhary Sai Aparna Aketi Gobinda Saha Kaushik Roy CLL 47 3 0 27 Mar 2023
Gossiped and Quantized Online Multi-Kernel Learning Tomàs Ortega Hamid Jafarkhani 27 5 0 24 Jan 2023
CEDAS: A Compressed Decentralized Stochastic Gradient Method with Improved Convergence Kun-Yen Huang Shin-Yi Pu 30 9 0 14 Jan 2023
Decentralized Stochastic Gradient Descent Ascent for Finite-Sum Minimax Problems Hongchang Gao 24 16 0 06 Dec 2022
PersA-FL: Personalized Asynchronous Federated Learning Taha Toghani Soomin Lee César A. Uribe FedML 34 6 0 03 Oct 2022
Unbounded Gradients in Federated Learning with Buffered Asynchronous Aggregation Taha Toghani César A. Uribe FedML 35 14 0 03 Oct 2022
On Generalization of Decentralized Learning with Separable Data Hossein Taheri Christos Thrampoulidis FedML 27 10 0 15 Sep 2022
Decentralized Gossip-Based Stochastic Bilevel Optimization over Communication Networks Shuoguang Yang Xuezhou Zhang Mengdi Wang 37 42 0 22 Jun 2022
Communication-Efficient Distributionally Robust Decentralized Learning Matteo Zecchin Marios Kountouris David Gesbert 22 9 0 31 May 2022
Data-heterogeneity-aware Mixing for Decentralized Learning Yatin Dandi Anastasia Koloskova Martin Jaggi Sebastian U. Stich 38 18 0 13 Apr 2022
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam Yucheng Lu Conglong Li Minjia Zhang Christopher De Sa Yuxiong He OffRL AI4CE 24 20 0 12 Feb 2022
Asynchronous Decentralized Learning over Unreliable Wireless Networks Eunjeong Jeong Matteo Zecchin Marios Kountouris 19 16 0 02 Feb 2022
BEER: Fast $O(1/T)$ Rate for Decentralized Nonconvex Optimization with Communication Compression Haoyu Zhao Boyue Li Zhize Li Peter Richtárik Yuejie Chi 24 48 0 31 Jan 2022
Distributed Adaptive Learning Under Communication Constraints Marco Carpentiero Vincenzo Matta Ali H. Sayed 29 17 0 03 Dec 2021
Exponential Graph is Provably Efficient for Decentralized Deep Training Bicheng Ying Kun Yuan Yiming Chen Hanbin Hu Pan Pan W. Yin FedML 39 83 0 26 Oct 2021
DiNNO: Distributed Neural Network Optimization for Multi-Robot Collaborative Learning Javier Yu Joseph A. Vincent Mac Schwager 46 35 0 17 Sep 2021
Decentralized Composite Optimization with Compression Yao Li Xiaorui Liu Jiliang Tang Ming Yan Kun Yuan 19 9 0 10 Aug 2021
Communication Efficiency in Federated Learning: Achievements and Challenges Osama Shahid Seyedamin Pouriyeh R. Parizi Quan Z. Sheng Gautam Srivastava Liang Zhao FedML 40 74 0 23 Jul 2021
Secure Distributed Training at Scale Eduard A. Gorbunov Alexander Borzunov Michael Diskin Max Ryabinin FedML 21 15 0 21 Jun 2021
Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices Max Ryabinin Eduard A. Gorbunov Vsevolod Plokhotnyuk Gennady Pekhimenko 35 32 0 04 Mar 2021
On the Utility of Gradient Compression in Distributed Training Systems Saurabh Agarwal Hongyi Wang Shivaram Venkataraman Dimitris Papailiopoulos 31 46 0 28 Feb 2021
IntSGD: Adaptive Floatless Compression of Stochastic Gradients Konstantin Mishchenko Bokun Wang D. Kovalev Peter Richtárik 69 14 0 16 Feb 2021
MARINA: Faster Non-Convex Distributed Learning with Compression Eduard A. Gorbunov Konstantin Burlachenko Zhize Li Peter Richtárik 39 109 0 15 Feb 2021
Communication-efficient Distributed Cooperative Learning with Compressed Beliefs Taha Toghani César A. Uribe 22 15 0 14 Feb 2021
Sparse-Push: Communication- & Energy-Efficient Decentralized Distributed Learning over Directed & Time-Varying Graphs with non-IID Datasets Sai Aparna Aketi Amandeep Singh J. Rabaey 21 10 0 10 Feb 2021
Consensus Control for Decentralized Deep Learning Lingjing Kong Tao R. Lin Anastasia Koloskova Martin Jaggi Sebastian U. Stich 19 75 0 09 Feb 2021
1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed Hanlin Tang Shaoduo Gan A. A. Awan Samyam Rajbhandari Conglong Li Xiangru Lian Ji Liu Ce Zhang Yuxiong He AI4CE 37 84 0 04 Feb 2021
Federated Learning over Wireless Device-to-Device Networks: Algorithms and Convergence Analysis Hong Xing Osvaldo Simeone Suzhi Bi 42 92 0 29 Jan 2021
Optimising cost vs accuracy of decentralised analytics in fog computing environments Lorenzo Valerio A. Passarella M. Conti 27 1 0 09 Dec 2020
On the Benefits of Multiple Gossip Steps in Communication-Constrained Decentralized Optimization Abolfazl Hashemi Anish Acharya Rudrajit Das H. Vikalo Sujay Sanghavi Inderjit Dhillon 17 7 0 20 Nov 2020
Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM Chaouki Ben Issaid Anis Elgabli Jihong Park M. Bennis Mérouane Debbah FedML 31 13 0 14 Sep 2020
On Communication Compression for Distributed Optimization on Heterogeneous Data Sebastian U. Stich 47 22 0 04 Sep 2020