signSGD: Compressed Optimisation for Non-Convex Problems

13 February 2018

Jeremy Bernstein

Yu-Xiang Wang

Kamyar Azizzadenesheli

Papers citing "signSGD: Compressed Optimisation for Non-Convex Problems"

50 / 189 papers shown

Title
Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation Divyansh Jhunjhunwala Ankur Mallick Advait Gadhikar S. Kadhe Gauri Joshi 16 10 0 14 Oct 2021
ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training Hui-Po Wang Sebastian U. Stich Yang He Mario Fritz FedML AI4CE 30 46 0 11 Oct 2021
Federated Learning via Plurality Vote Kai Yue Richeng Jin Chau-Wai Wong H. Dai FedML 24 8 0 06 Oct 2021
Bayesian AirComp with Sign-Alignment Precoding for Wireless Federated Learning Chanhoo Park Seunghoon Lee Namyoon Lee 29 5 0 14 Sep 2021
Fundamental limits of over-the-air optimization: Are analog schemes optimal? Shubham K. Jha Prathamesh Mayekar Himanshu Tyagi 24 7 0 11 Sep 2021
Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization Difan Zou Yuan Cao Yuanzhi Li Quanquan Gu MLT AI4CE 44 38 0 25 Aug 2021
A Hard Label Black-box Adversarial Attack Against Graph Neural Networks Jiaming Mu Binghui Wang Qi Li Kun Sun Mingwei Xu Zhuotao Liu AAML 23 33 0 21 Aug 2021
EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning S. Vargaftik Ran Ben-Basat Amit Portnoy Gal Mendelson Y. Ben-Itzhak Michael Mitzenmacher FedML 43 45 0 19 Aug 2021
Decentralized Composite Optimization with Compression Yao Li Xiaorui Liu Jiliang Tang Ming Yan Kun Yuan 19 9 0 10 Aug 2021
Evaluating Federated Learning for Intrusion Detection in Internet of Things: Review and Challenges Enrique Mármol Campos Pablo Fernández Saura Aurora González-Vidal José Luis Hernández Ramos Jorge Bernal Bernabé G. Baldini A. Gómez-Skarmeta 38 149 0 02 Aug 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 187 412 0 14 Jul 2021
BAGUA: Scaling up Distributed Learning with System Relaxations Shaoduo Gan Xiangru Lian Rui Wang Jianbin Chang Chengjun Liu ... Jiawei Jiang Binhang Yuan Sen Yang Ji Liu Ce Zhang 23 30 0 03 Jul 2021
LNS-Madam: Low-Precision Training in Logarithmic Number System using Multiplicative Weight Update Jiawei Zhao Steve Dai Rangharajan Venkatesan Brian Zimmer Mustafa Ali Xuan Li Brucek Khailany B. Dally Anima Anandkumar MQ 33 13 0 26 Jun 2021
Escaping Saddle Points with Compressed SGD Dmitrii Avdiukhin G. Yaroslavtsev 14 4 0 21 May 2021
DP-SIGNSGD: When Efficiency Meets Privacy and Robustness Lingjuan Lyu FedML AAML 27 19 0 11 May 2021
Slashing Communication Traffic in Federated Learning by Transmitting Clustered Model Updates Laizhong Cui Xiaoxin Su Yipeng Zhou Yi Pan FedML 32 36 0 10 May 2021
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges Mingzhe Chen Deniz Gündüz Kaibin Huang Walid Saad M. Bennis Aneta Vulgarakis Feljan H. Vincent Poor 38 401 0 05 Apr 2021
Federated Learning: A Signal Processing Perspective Tomer Gafni Nir Shlezinger Kobi Cohen Yonina C. Eldar H. Vincent Poor FedML 34 128 0 31 Mar 2021
DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation Wei Ping Fan Wu Yunhui Long Luka Rimanic Ce Zhang Bo-wen Li FedML 43 63 0 20 Mar 2021
Cloth Manipulation Planning on Basis of Mesh Representations with Incomplete Domain Knowledge and Voxel-to-Mesh Estimation S. Arnold Daisuke Tanaka Kimitoshi Yamazaki 22 4 0 15 Mar 2021
Efficient Randomized Subspace Embeddings for Distributed Optimization under a Communication Budget R. Saha Mert Pilanci Andrea J. Goldsmith 26 5 0 13 Mar 2021
On the Utility of Gradient Compression in Distributed Training Systems Saurabh Agarwal Hongyi Wang Shivaram Venkataraman Dimitris Papailiopoulos 31 46 0 28 Feb 2021
Wirelessly Powered Federated Edge Learning: Optimal Tradeoffs Between Convergence and Power Transfer Qunsong Zeng Yuqing Du Kaibin Huang 34 35 0 24 Feb 2021
IntSGD: Adaptive Floatless Compression of Stochastic Gradients Konstantin Mishchenko Bokun Wang D. Kovalev Peter Richtárik 72 14 0 16 Feb 2021
GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training Chen Zhu Renkun Ni Zheng Xu Kezhi Kong Yifan Jiang Tom Goldstein ODL 41 53 0 16 Feb 2021
Distributed Second Order Methods with Fast Rates and Compressed Communication Rustem Islamov Xun Qian Peter Richtárik 32 51 0 14 Feb 2021
Enabling Binary Neural Network Training on the Edge Erwei Wang James J. Davis Daniele Moro Piotr Zielinski Jia Jie Lim C. Coelho S. Chatterjee P. Cheung G. Constantinides MQ 20 24 0 08 Feb 2021
Federated Learning over Wireless Device-to-Device Networks: Algorithms and Convergence Analysis Hong Xing Osvaldo Simeone Suzhi Bi 42 92 0 29 Jan 2021
Time-Correlated Sparsification for Communication-Efficient Federated Learning Emre Ozfatura Kerem Ozfatura Deniz Gunduz FedML 38 47 0 21 Jan 2021
Bayesian Federated Learning over Wireless Networks Seunghoon Lee Chanhoo Park Songnam Hong Yonina C. Eldar Namyoon Lee 25 23 0 31 Dec 2020
FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training Y. Fu Haoran You Yang Katie Zhao Yue Wang Chaojian Li K. Gopalakrishnan Zhangyang Wang Yingyan Lin MQ 35 32 0 24 Dec 2020
Adaptive Precision Training for Resource Constrained Devices Tian Huang Tao Luo Qiufeng Wang 34 5 0 23 Dec 2020
Faster Non-Convex Federated Learning via Global and Local Momentum Rudrajit Das Anish Acharya Abolfazl Hashemi Sujay Sanghavi Inderjit S. Dhillon Ufuk Topcu FedML 35 82 0 07 Dec 2020
Design and Analysis of Uplink and Downlink Communications for Federated Learning Sihui Zheng Cong Shen Xiang Chen 39 140 0 07 Dec 2020
On the Benefits of Multiple Gossip Steps in Communication-Constrained Decentralized Optimization Abolfazl Hashemi Anish Acharya Rudrajit Das H. Vikalo Sujay Sanghavi Inderjit Dhillon 20 7 0 20 Nov 2020
Compression Boosts Differentially Private Federated Learning Raouf Kerkouche G. Ács C. Castelluccia P. Genevès FedML 22 29 0 10 Nov 2020
Improving Neural Network Training in Low Dimensional Random Bases Frithjof Gressmann Zach Eaton-Rosen Carlo Luschi 22 28 0 09 Nov 2020
Local SGD: Unified Theory and New Efficient Methods Eduard A. Gorbunov Filip Hanzely Peter Richtárik FedML 35 109 0 03 Nov 2020
FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers Zheng Chai Yujing Chen Ali Anwar Liang Zhao Yue Cheng Huzefa Rangwala FedML 21 121 0 12 Oct 2020
Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM Chaouki Ben Issaid Anis Elgabli Jihong Park M. Bennis Mérouane Debbah FedML 31 13 0 14 Sep 2020
On Communication Compression for Distributed Optimization on Heterogeneous Data Sebastian U. Stich 47 22 0 04 Sep 2020
Periodic Stochastic Gradient Descent with Momentum for Decentralized Training Hongchang Gao Heng-Chiao Huang 17 25 0 24 Aug 2020
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications Jihong Park S. Samarakoon Anis Elgabli Joongheon Kim M. Bennis Seong-Lyun Kim Mérouane Debbah 34 161 0 06 Aug 2020
Cluster-Based Cooperative Digital Over-the-Air Aggregation for Wireless Federated Edge Learning Ruichen Jiang Sheng Zhou FedML 14 17 0 03 Aug 2020
FetchSGD: Communication-Efficient Federated Learning with Sketching D. Rothchild Ashwinee Panda Enayat Ullah Nikita Ivkin Ion Stoica Vladimir Braverman Joseph E. Gonzalez Raman Arora FedML 17 361 0 15 Jul 2020
Federated Learning with Compression: Unified Analysis and Sharp Guarantees Farzin Haddadpour Mohammad Mahdi Kamani Aryan Mokhtari M. Mahdavi FedML 33 271 0 02 Jul 2020
MaxVA: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients Chenfei Zhu Yu Cheng Zhe Gan Furong Huang Jingjing Liu Tom Goldstein ODL 27 2 0 21 Jun 2020
rTop-k: A Statistical Estimation Approach to Distributed SGD L. P. Barnes Huseyin A. Inan Berivan Isik Ayfer Özgür 29 65 0 21 May 2020
Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks Zhishuai Guo Mingrui Liu Zhuoning Yuan Li Shen Wei Liu Tianbao Yang 33 42 0 05 May 2020
Communication Efficient Federated Learning with Energy Awareness over Wireless Networks Richeng Jin Xiaofan He H. Dai 36 25 0 15 Apr 2020