signSGD: Compressed Optimisation for Non-Convex Problems

13 February 2018

Jeremy Bernstein

Yu-Xiang Wang

Kamyar Azizzadenesheli

Papers citing "signSGD: Compressed Optimisation for Non-Convex Problems"

38 / 188 papers shown

Title
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 27 12 0 06 Mar 2020
Stochastic-Sign SGD for Federated Learning with Theoretical Guarantees Richeng Jin Yufan Huang Xiaofan He H. Dai Tianfu Wu FedML 22 63 0 25 Feb 2020
Communication-Efficient Edge AI: Algorithms and Systems Yuanming Shi Kai Yang Tao Jiang Jun Zhang Khaled B. Letaief GNN 17 326 0 22 Feb 2020
Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressor M. Safaryan Egor Shulgin Peter Richtárik 24 60 0 20 Feb 2020
LaProp: Separating Momentum and Adaptivity in Adam Liu Ziyin Zhikang T.Wang Masahito Ueda ODL 8 18 0 12 Feb 2020
On the distance between two neural networks and the stability of learning Jeremy Bernstein Arash Vahdat Yisong Yue Xuan Li ODL 200 57 0 09 Feb 2020
Communication Efficient Federated Learning over Multiple Access Channels Wei-Ting Chang Ravi Tandon FedML 8 44 0 23 Jan 2020
One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis Guangxu Zhu Yuqing Du Deniz Gunduz Kaibin Huang 36 308 0 16 Jan 2020
AdderNet: Do We Really Need Multiplications in Deep Learning? Hanting Chen Yunhe Wang Chunjing Xu Boxin Shi Chao Xu Qi Tian Chang Xu 18 194 0 31 Dec 2019
Randomized Reactive Redundancy for Byzantine Fault-Tolerance in Parallelized Learning Nirupam Gupta Nitin H. Vaidya FedML 30 8 0 19 Dec 2019
On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning Aritra Dutta El Houcine Bergou A. Abdelmoniem Chen-Yu Ho Atal Narayan Sahu Marco Canini Panos Kalnis 25 76 0 19 Nov 2019
Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning XINYAN DAI Xiao Yan Kaiwen Zhou Han Yang K. K. Ng James Cheng Yu Fan FedML 24 47 0 12 Nov 2019
Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning Anis Elgabli Jihong Park Amrit Singh Bedi Chaouki Ben Issaid M. Bennis Vaneet Aggarwal 24 67 0 23 Oct 2019
High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learning Yuqing Du Sheng Yang Kaibin Huang 29 99 0 09 Oct 2019
Sign-OPT: A Query-Efficient Hard-label Adversarial Attack Minhao Cheng Simranjit Singh Patrick H. Chen Pin-Yu Chen Sijia Liu Cho-Jui Hsieh AAML 124 219 0 24 Sep 2019
diffGrad: An Optimization Method for Convolutional Neural Networks S. Dubey Soumendu Chakraborty S. K. Roy Snehasis Mukherjee S. Singh B. B. Chaudhuri ODL 97 183 0 12 Sep 2019
Gradient Descent with Compressed Iterates Ahmed Khaled Peter Richtárik 21 22 0 10 Sep 2019
Robust and Communication-Efficient Collaborative Learning Amirhossein Reisizadeh Hossein Taheri Aryan Mokhtari Hamed Hassani Ramtin Pedarsani 17 89 0 24 Jul 2019
Federated Learning over Wireless Fading Channels M. Amiri Deniz Gunduz 27 505 0 23 Jul 2019
QUOTIENT: Two-Party Secure Neural Network Training and Prediction Nitin Agrawal Ali Shahin Shamsabadi Matt J. Kusner Adria Gascon 22 212 0 08 Jul 2019
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization Thijs Vogels Sai Praneeth Karimireddy Martin Jaggi 17 316 0 31 May 2019
Limitations of the Empirical Fisher Approximation for Natural Gradient Descent Frederik Kunstner Lukas Balles Philipp Hennig 21 207 0 29 May 2019
Natural Compression for Distributed Deep Learning Samuel Horváth Chen-Yu Ho L. Horvath Atal Narayan Sahu Marco Canini Peter Richtárik 21 150 0 27 May 2019
LAGC: Lazily Aggregated Gradient Coding for Straggler-Tolerant and Communication-Efficient Distributed Learning Jingjing Zhang Osvaldo Simeone 18 31 0 22 May 2019
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes Yang You Jing Li Sashank J. Reddi Jonathan Hseu Sanjiv Kumar Srinadh Bhojanapalli Xiaodan Song J. Demmel Kurt Keutzer Cho-Jui Hsieh ODL 28 978 0 01 Apr 2019
Robust and Communication-Efficient Federated Learning from Non-IID Data Felix Sattler Simon Wiedemann K. Müller Wojciech Samek FedML 24 1,330 0 07 Mar 2019
Distributed Learning with Sublinear Communication Jayadev Acharya Christopher De Sa Dylan J. Foster Karthik Sridharan FedML 21 40 0 28 Feb 2019
On Maintaining Linear Convergence of Distributed Learning and Optimization under Limited Communication Sindri Magnússon H. S. Ghadikolaei Na Li 19 81 0 26 Feb 2019
Quantized Frank-Wolfe: Faster Optimization, Lower Communication, and Projection Free Mingrui Zhang Lin Chen Aryan Mokhtari Hamed Hassani Amin Karbasi 16 8 0 17 Feb 2019
99% of Distributed Optimization is a Waste of Time: The Issue and How to Fix it Konstantin Mishchenko Filip Hanzely Peter Richtárik 16 13 0 27 Jan 2019
A Distributed Synchronous SGD Algorithm with Global Top- $k$ Sparsification for Low Bandwidth Networks S. Shi Qiang-qiang Wang Kaiyong Zhao Zhenheng Tang Yuxin Wang Xiang Huang Xiaowen Chu 32 134 0 14 Jan 2019
A Sufficient Condition for Convergences of Adam and RMSProp Fangyu Zou Li Shen Zequn Jie Weizhong Zhang Wei Liu 27 364 0 23 Nov 2018
Policy Gradient in Partially Observable Environments: Approximation and Convergence Kamyar Azizzadenesheli Manish Kumar Bera Anima Anandkumar OffRL 22 8 0 18 Oct 2018
signSGD with Majority Vote is Communication Efficient And Fault Tolerant Jeremy Bernstein Jiawei Zhao Kamyar Azizzadenesheli Anima Anandkumar FedML 23 46 0 11 Oct 2018
Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms Jianyu Wang Gauri Joshi 18 348 0 22 Aug 2018
Don't Use Large Mini-Batches, Use Local SGD Tao R. Lin Sebastian U. Stich Kumar Kshitij Patel Martin Jaggi 54 429 0 22 Aug 2018
Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communication Felix Sattler Simon Wiedemann K. Müller Wojciech Samek MQ 11 210 0 22 May 2018
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark W. Schmidt 139 1,199 0 16 Aug 2016