Sparse Communication for Distributed Gradient Descent

17 April 2017

Kenneth Heafield

Papers citing "Sparse Communication for Distributed Gradient Descent"

50 / 142 papers shown

Title
Privacy Preserving Conversion Modeling in Data Clean Room Kungang Li Xiangyi Chen Ling Leng Jiajing Xu Jiankai Sun Behnam Rezaei 12 1 0 20 May 2025
$γ$ -FedHT: Stepsize-Aware Hard-Threshold Gradient Compression in Federated Learning Rongwei Lu Yutong Jiang Jinrui Zhang Chunyang Li Yifei Zhu Bin Chen Zhi Wang FedML 16 0 0 18 May 2025
Communication-Efficient Federated Fine-Tuning of Language Models via Dynamic Update Schedules Michail Theologitis V. Samoladas Antonios Deligiannakis 34 0 0 07 May 2025
Sparsification Under Siege: Defending Against Poisoning Attacks in Communication-Efficient Federated Learning Zhiyong Jin Runhua Xu Chong Li Yunxing Liu Jianxin Li AAML FedML 48 0 0 30 Apr 2025
Biased Federated Learning under Wireless Heterogeneity Muhammad Faraz Ul Abrar Nicolò Michelusi FedML 51 0 0 08 Mar 2025
Heterogeneity Matters even More in Distributed Learning: Study from Generalization Perspective Masoud Kavian Romain Chor Milad Sefidgaran Abdellatif Zaidi FedML 55 1 0 03 Mar 2025
CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling Kaiyuan Zhang Siyuan Cheng Guangyu Shen Bruno Ribeiro Shengwei An Pin-Yu Chen Xinming Zhang Ninghui Li 154 1 0 28 Jan 2025
Trustworthiness of Stochastic Gradient Descent in Distributed Learning Hongyang Li Caesar Wu Mohammed Chadli Said Mammar Pascal Bouvry 56 1 0 28 Oct 2024
No Need to Talk: Asynchronous Mixture of Language Models Anastasiia Filippova Angelos Katharopoulos David Grangier Ronan Collobert MoE 46 0 0 04 Oct 2024
Novel Gradient Sparsification Algorithm via Bayesian Inference Ali Bereyhi B. Liang G. Boudreau Ali Afana 36 2 0 23 Sep 2024
Leakage-Resilient and Carbon-Neutral Aggregation Featuring the Federated AI-enabled Critical Infrastructure Zehang Deng Ruoxi Sun Minhui Xue Sheng Wen S. Çamtepe Surya Nepal Yang Xiang 52 1 0 24 May 2024
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey Feng Liang Zhen Zhang Haifeng Lu Victor C. M. Leung Yanyi Guo Xiping Hu GNN 39 6 0 09 Apr 2024
RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretation Weiying Xie Zixuan Wang Jitao Ma Daixun Li Yunsong Li 32 0 0 29 Dec 2023
Sparse Training for Federated Learning with Regularized Error Correction Ran Greidi Kobi Cohen FedML 35 1 0 21 Dec 2023
FedDIP: Federated Learning with Extreme Dynamic Pruning and Incremental Regularization Qianyu Long Christos Anagnostopoulos S. P. Parambath Daning Bi AI4CE FedML 23 2 0 13 Sep 2023
FedDD: Toward Communication-efficient Federated Learning with Differential Parameter Dropout Zhiying Feng Xu Chen Qiong Wu Wenhua Wu Xiaoxi Zhang Qian Huang FedML 41 2 0 31 Aug 2023
Revolutionizing Wireless Networks with Federated Learning: A Comprehensive Review Sajjad Emdadi Mahdimahalleh AI4CE 40 0 0 01 Aug 2023
Compressed Private Aggregation for Scalable and Robust Federated Learning over Massive Networks Natalie Lang Nir Shlezinger Rafael G. L. DÓliveira S. E. Rouayheb FedML 75 4 0 01 Aug 2023
Get More for Less in Decentralized Learning Systems Akash Dhasade Anne-Marie Kermarrec Rafael Pires Rishi Sharma Milos Vujasinovic Jeffrey Wigger 34 7 0 07 Jun 2023
Federated Neural Radiance Fields Lachlan Holden Feras Dayoub D. Harvey Tat-Jun Chin FedML AI4CE 40 4 0 02 May 2023
Towards Accurate Post-Training Quantization for Vision Transformer Yifu Ding Haotong Qin Qing-Yu Yan Z. Chai Junjie Liu Xiaolin K. Wei Xianglong Liu MQ 54 69 0 25 Mar 2023
Cloudless-Training: A Framework to Improve Efficiency of Geo-Distributed ML Training W. Tan Xiao Shi Cunchi Lv Xiaofang Zhao FedML 31 1 0 09 Mar 2023
FedREP: A Byzantine-Robust, Communication-Efficient and Privacy-Preserving Framework for Federated Learning Yi-Rui Yang Kun Wang Wulu Li FedML 44 3 0 09 Mar 2023
Communication-efficient Federated Learning with Single-Step Synthetic Features Compressor for Faster Convergence Yuhao Zhou Mingjia Shi Yuanxi Li Qing Ye Yanan Sun Jiancheng Lv 21 4 0 27 Feb 2023
DISCO: Distributed Inference with Sparse Communications Minghai Qin Chaowen Sun Jaco A. Hofmann D. Vučinić FedML 27 1 0 22 Feb 2023
Sparse-SignSGD with Majority Vote for Communication-Efficient Distributed Learning Chanho Park Namyoon Lee FedML 35 3 0 15 Feb 2023
FedPass: Privacy-Preserving Vertical Federated Deep Learning with Adaptive Obfuscation Hanlin Gu Jiahuan Luo Yan Kang Lixin Fan Qiang Yang FedML 36 13 0 30 Jan 2023
M22: A Communication-Efficient Algorithm for Federated Learning Inspired by Rate-Distortion Yangyi Liu Stefano Rini Sadaf Salehkalaibar Jun Chen FedML 21 4 0 23 Jan 2023
Does compressing activations help model parallel training? S. Bian Dacheng Li Hongyi Wang Eric P. Xing Shivaram Venkataraman 35 5 0 06 Jan 2023
A Theory of I/O-Efficient Sparse Neural Network Inference Niels Gleinig Tal Ben-Nun Torsten Hoefler 27 0 0 03 Jan 2023
Mutual Information Regularization for Vertical Federated Learning Tianyuan Zou Yang Liu Ya-Qin Zhang AAML FedML 35 7 0 01 Jan 2023
Deep Hierarchy Quantization Compression algorithm based on Dynamic Sampling W. Jiang Gang Liu Xiaofeng Chen Yipeng Zhou FedML 19 0 0 30 Dec 2022
Graph Federated Learning for CIoT Devices in Smart Home Applications Arash Rasti-Meymandi S. M. Sheikholeslami J. Abouei Konstantinos N. Plataniotis FedML 29 18 0 29 Dec 2022
Communication-Efficient Federated Learning for Heterogeneous Edge Devices Based on Adaptive Gradient Quantization Heting Liu Fang He Guohong Cao FedML MQ 35 24 0 16 Dec 2022
Refiner: Data Refining against Gradient Leakage Attacks in Federated Learning Mingyuan Fan Cen Chen Chengyu Wang Ximeng Liu Wenmeng Zhou Jun Huang AAML FedML 34 0 0 05 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 36 4 0 25 Nov 2022
Vertical Federated Learning: Concepts, Advances and Challenges Yang Liu Yan Kang Tianyuan Zou Yanhong Pu Yuanqin He Xiaozhou Ye Ye Ouyang Yaqin Zhang Qian Yang FedML 64 162 0 23 Nov 2022
Improving Federated Learning Communication Efficiency with Global Momentum Fusion for Gradient Compression Schemes Chun-Chih Kuo Ted T. Kuo Chia-Yu Lin FedML 18 1 0 17 Nov 2022
Efficient and Light-Weight Federated Learning via Asynchronous Distributed Dropout Chen Dun Mirian Hipolito Garcia C. Jermaine Dimitrios Dimitriadis Anastasios Kyrillidis 66 20 0 28 Oct 2022
Learning to Invert: Simple Adaptive Attacks for Gradient Inversion in Federated Learning Ruihan Wu Xiangyu Chen Chuan Guo Kilian Q. Weinberger FedML 20 26 0 19 Oct 2022
Communication-Efficient Adam-Type Algorithms for Distributed Data Mining Wenhan Xian Feihu Huang Heng-Chiao Huang FedML 35 0 0 14 Oct 2022
Downlink Compression Improves TopK Sparsification William Zou H. Sterck Jun Liu 21 0 0 30 Sep 2022
Joint Privacy Enhancement and Quantization in Federated Learning Natalie Lang Elad Sofer Tomer Shaked Nir Shlezinger FedML 39 46 0 23 Aug 2022
A Fast Blockchain-based Federated Learning Framework with Compressed Communications Laizhong Cui Xiaoxin Su Yipeng Zhou FedML 22 23 0 12 Aug 2022
Towards Efficient Communications in Federated Learning: A Contemporary Survey Zihao Zhao Yuzhu Mao Yang Liu Linqi Song Ouyang Ye Xinlei Chen Wenbo Ding FedML 59 60 0 02 Aug 2022
BiFeat: Supercharge GNN Training via Graph Feature Quantization Yuxin Ma Ping Gong Jun Yi Z. Yao Cheng-rong Li Yuxiong He Feng Yan GNN 21 6 0 29 Jul 2022
Fundamental Limits of Communication Efficiency for Model Aggregation in Distributed Learning: A Rate-Distortion Approach Naifu Zhang M. Tao Jia Wang Fan Xu 19 13 0 28 Jun 2022
sqSGD: Locally Private and Communication Efficient Federated Learning Yan Feng Tao Xiong Ruofan Wu Lingjuan Lv Leilei Shi FedML 31 2 0 21 Jun 2022
Gradient Obfuscation Gives a False Sense of Security in Federated Learning Kai Yue Richeng Jin Chau-Wai Wong D. Baron H. Dai FedML 36 46 0 08 Jun 2022
Communication-Efficient Distributionally Robust Decentralized Learning Matteo Zecchin Marios Kountouris David Gesbert 25 9 0 31 May 2022