Error Feedback Fixes SignSGD and other Gradient Compression Schemes

28 January 2019

Sai Praneeth Karimireddy

Papers citing "Error Feedback Fixes SignSGD and other Gradient Compression Schemes"

50 / 104 papers shown

Title
Convergence Analysis of Asynchronous Federated Learning with Gradient Compression for Non-Convex Optimization Diying Yang Yingwei Hou Danyang Xiao Weigang Wu FedML 39 0 0 28 Apr 2025
Accelerated Distributed Optimization with Compression and Error Feedback Yuan Gao Anton Rodomanov Jeremy Rack Sebastian U. Stich 51 0 0 11 Mar 2025
On the Byzantine Fault Tolerance of signSGD with Majority Vote Emanuele Mengoli Luzius Moll Virgilio Strozzi El-Mahdi El-Mhamdi AAML FedML 67 0 0 26 Feb 2025
Distributed Sign Momentum with Local Steps for Training Transformers Shuhua Yu Ding Zhou Cong Xie An Xu Zhi-Li Zhang Xin Liu S. Kar 69 0 0 26 Nov 2024
Trustworthiness of Stochastic Gradient Descent in Distributed Learning Hongyang Li Caesar Wu Mohammed Chadli Said Mammar Pascal Bouvry 56 1 0 28 Oct 2024
LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics Thomas Robert M. Safaryan Ionut-Vlad Modoranu Dan Alistarh ODL 36 2 0 21 Oct 2024
From Gradient Clipping to Normalization for Heavy Tailed SGD Florian Hübler Ilyas Fatkhullin Niao He 40 5 0 17 Oct 2024
Communication-efficient Vertical Federated Learning via Compressed Error Feedback Pedro Valdeira João Xavier Cláudia Soares Yuejie Chi FedML 45 4 0 20 Jun 2024
BOLD: Boolean Logic Deep Learning Van Minh Nguyen Cristian Ocampo Aymen Askri Louis Leconte Ba-Hien Tran AI4CE 40 0 0 25 May 2024
SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data Sakshi Choudhary Sai Aparna Aketi Kaushik Roy FedML 45 0 0 22 May 2024
SignSGD with Federated Voting Chanho Park H. Vincent Poor Namyoon Lee FedML 40 1 0 25 Mar 2024
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretation Weiying Xie Zixuan Wang Jitao Ma Daixun Li Yunsong Li 30 0 0 29 Dec 2023
Sparse Training for Federated Learning with Regularized Error Correction Ran Greidi Kobi Cohen FedML 30 1 0 21 Dec 2023
Straggler-resilient Federated Learning: Tackling Computation Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Network Student Member Ieee Hongda Wu F. I. C. V. Ping Wang Aswartha Narayana FedML 52 1 0 16 Nov 2023
AirFL-Mem: Improving Communication-Learning Trade-Off by Long-Term Memory Haifeng Wen Hong Xing Osvaldo Simeone 40 0 0 25 Oct 2023
Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization Zhen Qin Zhishuai Liu Pan Xu 23 1 0 24 Oct 2023
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates Ahmad Rammal Kaja Gruntkowska Nikita Fedin Eduard A. Gorbunov Peter Richtárik 45 5 0 15 Oct 2023
Asynchronous Federated Learning with Bidirectional Quantized Communications and Buffered Aggregation Tomàs Ortega Hamid Jafarkhani FedML 30 6 0 01 Aug 2023
Revolutionizing Wireless Networks with Federated Learning: A Comprehensive Review Sajjad Emdadi Mahdimahalleh AI4CE 35 0 0 01 Aug 2023
Clip21: Error Feedback for Gradient Clipping Sarit Khirirat Eduard A. Gorbunov Samuel Horváth Rustem Islamov Fakhri Karray Peter Richtárik 32 10 0 30 May 2023
Error Feedback Shines when Features are Rare Peter Richtárik Elnur Gasanov Konstantin Burlachenko 36 2 0 24 May 2023
Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods Junchi Yang Xiang Li Ilyas Fatkhullin Niao He 39 15 0 21 May 2023
GraVAC: Adaptive Compression for Communication-Efficient Distributed DL Training S. Tyagi Martin Swany 25 4 0 20 May 2023
Communication and Energy Efficient Wireless Federated Learning with Intrinsic Privacy Zhenxiao Zhang Yuanxiong Guo Yuguang Fang Yanmin Gong 36 4 0 15 Apr 2023
Communication-efficient Federated Learning with Single-Step Synthetic Features Compressor for Faster Convergence Yuhao Zhou Mingjia Shi Yuanxi Li Qing Ye Yanan Sun Jiancheng Lv 18 3 0 27 Feb 2023
DoCoFL: Downlink Compression for Cross-Device Federated Learning Ron Dorfman S. Vargaftik Y. Ben-Itzhak Kfir Y. Levy FedML 32 19 0 01 Feb 2023
A Comprehensive Survey of Dataset Distillation Shiye Lei Dacheng Tao DD 31 88 0 13 Jan 2023
Federated Learning with Flexible Control Shiqiang Wang Jake B. Perazzone Mingyue Ji Kevin S. Chan FedML 28 17 0 16 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 34 4 0 25 Nov 2022
Fast Adaptive Federated Bilevel Optimization Feihu Huang FedML 20 7 0 02 Nov 2022
Adaptive Compression for Communication-Efficient Distributed Training Maksim Makarenko Elnur Gasanov Rustem Islamov Abdurakhmon Sadiev Peter Richtárik 36 12 0 31 Oct 2022
L-GreCo: Layerwise-Adaptive Gradient Compression for Efficient and Accurate Deep Learning Mohammadreza Alimohammadi I. Markov Elias Frantar Dan Alistarh 35 5 0 31 Oct 2022
Coresets for Vertical Federated Learning: Regularized Linear Regression and $K$ -Means Clustering Lingxiao Huang Zhize Li Jialin Sun Haoyu Zhao FedML 44 9 0 26 Oct 2022
FedGRec: Federated Graph Recommender System with Lazy Update of Latent Embeddings Junyi Li Heng-Chiao Huang FedML 24 6 0 25 Oct 2022
Communication-Efficient Adam-Type Algorithms for Distributed Data Mining Wenhan Xian Feihu Huang Heng-Chiao Huang FedML 35 0 0 14 Oct 2022
Downlink Compression Improves TopK Sparsification William Zou H. Sterck Jun Liu 21 0 0 30 Sep 2022
Joint Privacy Enhancement and Quantization in Federated Learning Natalie Lang Elad Sofer Tomer Shaked Nir Shlezinger FedML 37 46 0 23 Aug 2022
Efficient-Adam: Communication-Efficient Distributed Adam Congliang Chen Li Shen Wei Liu Zhi-Quan Luo 34 19 0 28 May 2022
Communication-Efficient Adaptive Federated Learning Yujia Wang Lu Lin Jinghui Chen FedML 27 71 0 05 May 2022
FedVQCS: Federated Learning via Vector Quantized Compressed Sensing Yong-Nam Oh Yo-Seb Jeon Mingzhe Chen Walid Saad FedML 30 10 0 16 Apr 2022
Convert, compress, correct: Three steps toward communication-efficient DNN training Zhongzhu Chen Eduin E. Hernandez Yu-Chih Huang Stefano Rini 20 0 0 17 Mar 2022
Linear Stochastic Bandits over a Bit-Constrained Channel A. Mitra Hamed Hassani George J. Pappas 39 8 0 02 Mar 2022
Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methods Aleksandr Beznosikov Eduard A. Gorbunov Hugo Berard Nicolas Loizou 19 48 0 15 Feb 2022
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam Yucheng Lu Conglong Li Minjia Zhang Christopher De Sa Yuxiong He OffRL AI4CE 24 20 0 12 Feb 2022
Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank? Sheikh Shams Azam Seyyedali Hosseinalipour Qiang Qiu Christopher G. Brinton FedML 26 20 0 01 Feb 2022
BEER: Fast $O(1/T)$ Rate for Decentralized Nonconvex Optimization with Communication Compression Haoyu Zhao Boyue Li Zhize Li Peter Richtárik Yuejie Chi 29 48 0 31 Jan 2022
Decentralized Multi-Task Stochastic Optimization With Compressed Communications Navjot Singh Xuanyu Cao Suhas Diggavi Tamer Basar 18 9 0 23 Dec 2021
SparseFed: Mitigating Model Poisoning Attacks in Federated Learning with Sparsification Ashwinee Panda Saeed Mahloujifar A. Bhagoji Supriyo Chakraborty Prateek Mittal FedML AAML 17 84 0 12 Dec 2021
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient Xiaoge Deng Dongsheng Li Tao Sun Xicheng Lu FedML 26 0 0 08 Dec 2021
Collaborative Learning over Wireless Networks: An Introductory Overview Emre Ozfatura Deniz Gunduz H. Vincent Poor 27 11 0 07 Dec 2021