signSGD: Compressed Optimisation for Non-Convex Problems

13 February 2018

Jeremy Bernstein

Yu-Xiang Wang

Kamyar Azizzadenesheli

Papers citing "signSGD: Compressed Optimisation for Non-Convex Problems"

50 / 188 papers shown

Title
Breaking the Communication-Privacy-Accuracy Tradeoff with $f$ -Differential Privacy Richeng Jin Z. Su C. Zhong Zhaoyang Zhang Tony Q. S. Quek H. Dai FedML 29 2 0 19 Feb 2023
Symbolic Discovery of Optimization Algorithms Xiangning Chen Chen Liang Da Huang Esteban Real Kaiyuan Wang ... Xuanyi Dong Thang Luong Cho-Jui Hsieh Yifeng Lu Quoc V. Le 64 352 0 13 Feb 2023
DoCoFL: Downlink Compression for Cross-Device Federated Learning Ron Dorfman S. Vargaftik Y. Ben-Itzhak Kfir Y. Levy FedML 24 18 0 01 Feb 2023
PolarAir: A Compressed Sensing Scheme for Over-the-Air Federated Learning Michail Gkagkos Krishna R. Narayanan J. Chamberland C. Georghiades 40 0 0 24 Jan 2023
Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression Jaeyong Song Jinkyu Yim Jaewon Jung Hongsun Jang H. Kim Youngsok Kim Jinho Lee GNN 24 25 0 24 Jan 2023
M22: A Communication-Efficient Algorithm for Federated Learning Inspired by Rate-Distortion Yangyi Liu Stefano Rini Sadaf Salehkalaibar Jun Chen FedML 21 4 0 23 Jan 2023
CEDAS: A Compressed Decentralized Stochastic Gradient Method with Improved Convergence Kun-Yen Huang Shin-Yi Pu 30 9 0 14 Jan 2023
A Comprehensive Survey of Dataset Distillation Shiye Lei Dacheng Tao DD 31 87 0 13 Jan 2023
Does compressing activations help model parallel training? S. Bian Dacheng Li Hongyi Wang Eric P. Xing Shivaram Venkataraman 19 5 0 06 Jan 2023
Efficient On-device Training via Gradient Filtering Yuedong Yang Guihong Li R. Marculescu 31 18 0 01 Jan 2023
Federated Learning with Flexible Control Shiqiang Wang Jake B. Perazzone Mingyue Ji Kevin S. Chan FedML 28 17 0 16 Dec 2022
Refiner: Data Refining against Gradient Leakage Attacks in Federated Learning Mingyuan Fan Cen Chen Chengyu Wang Ximeng Liu Wenmeng Zhou Jun Huang AAML FedML 34 0 0 05 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 32 4 0 25 Nov 2022
Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms Vincent Plassier Alain Durmus Eric Moulines FedML 21 6 0 31 Oct 2022
An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization Elvin Lo Pin-Yu Chen 37 0 0 27 Oct 2022
FeDXL: Provable Federated Learning for Deep X-Risk Optimization Zhishuai Guo R. L. Jin Jiebo Luo Tianbao Yang FedML 47 8 0 26 Oct 2022
Learning to Invert: Simple Adaptive Attacks for Gradient Inversion in Federated Learning Ruihan Wu Xiangyu Chen Chuan Guo Kilian Q. Weinberger FedML 12 26 0 19 Oct 2022
Over-the-Air Computation Based on Balanced Number Systems for Federated Edge Learning Alphan Șahin FedML 28 14 0 13 Oct 2022
Over-the-Air Computation over Balanced Numerals Alphan Șahin Rui Yang 23 9 0 22 Sep 2022
A Demonstration of Over-the-Air Computation for Federated Edge Learning Alphan Șahin 17 7 0 20 Sep 2022
Joint Privacy Enhancement and Quantization in Federated Learning Natalie Lang Elad Sofer Tomer Shaked Nir Shlezinger FedML 34 46 0 23 Aug 2022
MUDGUARD: Taming Malicious Majorities in Federated Learning using Privacy-Preserving Byzantine-Robust Clustering Rui Wang Xingkai Wang H. Chen Jérémie Decouchant S. Picek Ziqiang Liu K. Liang 36 1 0 22 Aug 2022
Practical Vertical Federated Learning with Unsupervised Representation Learning Zhaomin Wu Yue Liu Bingsheng He FedML 30 37 0 13 Aug 2022
Towards Efficient Communications in Federated Learning: A Contemporary Survey Zihao Zhao Yuzhu Mao Yang Liu Linqi Song Ouyang Ye Xinlei Chen Wenbo Ding FedML 51 59 0 02 Aug 2022
Fundamental Limits of Communication Efficiency for Model Aggregation in Distributed Learning: A Rate-Distortion Approach Naifu Zhang M. Tao Jia Wang Fan Xu 11 13 0 28 Jun 2022
Distributed Adversarial Training to Robustify Deep Neural Networks at Scale Gaoyuan Zhang Songtao Lu Yihua Zhang Xiangyi Chen Pin-Yu Chen Quanfu Fan Lee Martie L. Horesh Min-Fong Hong Sijia Liu OOD 24 12 0 13 Jun 2022
Gradient Obfuscation Gives a False Sense of Security in Federated Learning Kai Yue Richeng Jin Chau-Wai Wong D. Baron H. Dai FedML 31 46 0 08 Jun 2022
Communication-Efficient Distributionally Robust Decentralized Learning Matteo Zecchin Marios Kountouris David Gesbert 20 9 0 31 May 2022
Efficient-Adam: Communication-Efficient Distributed Adam Congliang Chen Li Shen Wei Liu Zhi-Quan Luo 25 19 0 28 May 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 27 10 0 08 May 2022
Communication-Efficient Adaptive Federated Learning Yujia Wang Lu Lin Jinghui Chen FedML 24 71 0 05 May 2022
FedVQCS: Federated Learning via Vector Quantized Compressed Sensing Yong-Nam Oh Yo-Seb Jeon Mingzhe Chen Walid Saad FedML 22 10 0 16 Apr 2022
Enabling All In-Edge Deep Learning: A Literature Review Praveen Joshi Mohammed Hasanuzzaman Chandra Thapa Haithem Afli T. Scully 28 22 0 07 Apr 2022
Nonlinear gradient mappings and stochastic optimization: A general framework with applications to heavy-tail noise D. Jakovetić Dragana Bajović Anit Kumar Sahu S. Kar Nemanja Milošević Dusan Stamenkovic 17 12 0 06 Apr 2022
FedSynth: Gradient Compression via Synthetic Data in Federated Learning Shengyuan Hu Jack Goetz Kshitiz Malik Hongyuan Zhan Zhe Liu Yue Liu DD FedML 37 38 0 04 Apr 2022
Over-the-Air Federated Learning via Second-Order Optimization Peng Yang Yuning Jiang Ting Wang Yong Zhou Yuanming Shi Colin N. Jones 45 28 0 29 Mar 2022
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam Yucheng Lu Conglong Li Minjia Zhang Christopher De Sa Yuxiong He OffRL AI4CE 24 20 0 12 Feb 2022
SignSGD: Fault-Tolerance to Blind and Byzantine Adversaries J. Akoun S. Meyer AAML FedML 14 1 0 04 Feb 2022
Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank? Sheikh Shams Azam Seyyedali Hosseinalipour Qiang Qiu Christopher G. Brinton FedML 20 20 0 01 Feb 2022
A Stochastic Bundle Method for Interpolating Networks Alasdair Paren Leonard Berrada Rudra P. K. Poudel M. P. Kumar 24 4 0 29 Jan 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 31 8 0 09 Jan 2022
Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats Brian Chmiel Ron Banner Elad Hoffer Hilla Ben Yaacov Daniel Soudry MQ 25 22 0 19 Dec 2021
Federated Two-stage Learning with Sign-based Voting Zichen Ma Zihan Lu Yu Lu Wenye Li Jinfeng Yi Shuguang Cui FedML 33 2 0 10 Dec 2021
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient Xiaoge Deng Dongsheng Li Tao Sun Xicheng Lu FedML 26 0 0 08 Dec 2021
Improving Differentially Private SGD via Randomly Sparsified Gradients Junyi Zhu Matthew B. Blaschko 26 5 0 01 Dec 2021
Communication-Efficient Federated Learning via Quantized Compressed Sensing Yong-Nam Oh Namyoon Lee Yo-Seb Jeon H. Vincent Poor FedML MQ 22 34 0 30 Nov 2021
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications Khaled B. Letaief Yuanming Shi Jianmin Lu Jianhua Lu 34 416 0 24 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 27 14 0 01 Nov 2021
BEV-SGD: Best Effort Voting SGD for Analog Aggregation Based Federated Learning against Byzantine Attackers Xin-Yue Fan Yue Wang Yan Huo Zhi Tian FedML 22 23 0 18 Oct 2021
Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation Divyansh Jhunjhunwala Ankur Mallick Advait Gadhikar S. Kadhe Gauri Joshi 16 10 0 14 Oct 2021