On Distributed Adaptive Optimization with Gradient Compression

11 May 2022

Papers citing "On Distributed Adaptive Optimization with Gradient Compression"

17 / 17 papers shown

Title
Convergence Analysis of Asynchronous Federated Learning with Gradient Compression for Non-Convex Optimization Diying Yang Yingwei Hou Danyang Xiao Weigang Wu FedML 39 0 0 28 Apr 2025
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 37 0 0 11 Nov 2024
LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics Thomas Robert M. Safaryan Ionut-Vlad Modoranu Dan Alistarh ODL 36 2 0 21 Oct 2024
FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression Zhenheng Tang Xueze Kang Yiming Yin Xinglin Pan Yuxin Wang ... Shaohuai Shi Amelie Chi Zhou Bo Li Bingsheng He Xiaowen Chu AI4CE 81 8 0 16 Oct 2024
Studying the Impact of TensorFlow and PyTorch Bindings on Machine Learning Software Quality Hao Li Gopi Krishnan Rajbahadur C. Bezemer 39 5 0 07 Jul 2024
MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence Ionut-Vlad Modoranu M. Safaryan Grigory Malinovsky Eldar Kurtic Thomas Robert Peter Richtárik Dan Alistarh MQ 42 12 0 24 May 2024
FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs Zhenheng Tang Yuxin Wang Xin He Longteng Zhang Xinglin Pan ... Rongfei Zeng Kaiyong Zhao S. Shi Bingsheng He Xiaowen Chu 41 30 0 03 Sep 2023
Stochastic Controlled Averaging for Federated Learning with Communication Compression Xinmeng Huang Ping Li Xiaoyun Li 43 198 0 16 Aug 2023
Error Feedback Can Accurately Compress Preconditioners Ionut-Vlad Modoranu A. Kalinov Eldar Kurtic Elias Frantar Dan Alistarh ODL 14 4 0 09 Jun 2023
Momentum Provably Improves Error Feedback! Ilyas Fatkhullin A. Tyurin Peter Richtárik 28 19 0 24 May 2023
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks Hao Sun Li Shen Qihuang Zhong Liang Ding Shi-Yong Chen Jingwei Sun Jing Li Guangzhong Sun Dacheng Tao 49 31 0 01 Mar 2023
THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression Minghao Li Ran Ben-Basat S. Vargaftik Chon-In Lao Ke Xu Michael Mitzenmacher Minlan Yu Harvard University 26 15 0 16 Feb 2023
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 34 4 0 25 Nov 2022
On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyond Xiao-Tong Yuan P. Li FedML 18 59 0 10 Jun 2022
Layer-wise and Dimension-wise Locally Adaptive Federated Learning Belhal Karimi Ping Li Xiaoyun Li FedML 42 3 0 01 Oct 2021
On the Convergence of Decentralized Adaptive Gradient Methods Xiangyi Chen Belhal Karimi Weijie Zhao Ping Li 23 21 0 07 Sep 2021
Communication-Efficient Distributed Deep Learning: A Comprehensive Survey Zhenheng Tang S. Shi Wei Wang Bo-wen Li Xiaowen Chu 21 48 0 10 Mar 2020