On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants

23 June 2015

Papers citing "On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants"

35 / 35 papers shown

Title
An Efficient Row-Based Sparse Fine-Tuning Cen-Jhih Li Aditya Bhaskara 56 0 0 17 Feb 2025
SYNTHESIS: A Semi-Asynchronous Path-Integrated Stochastic Gradient Method for Distributed Learning in Computing Clusters Zhuqing Liu Xin Zhang Jia-Wei Liu 30 1 0 17 Aug 2022
Byzantine Fault Tolerance in Distributed Machine Learning : a Survey Djamila Bouhata Hamouma Moumen Moumen Hamouma Ahcène Bounceur AI4CE 27 7 0 05 May 2022
Federated Learning with Buffered Asynchronous Aggregation John Nguyen Kshitiz Malik Hongyuan Zhan Ashkan Yousefpour Michael G. Rabbat Mani Malek Dzmitry Huba FedML 33 288 0 11 Jun 2021
Variance Reduction via Accelerated Dual Averaging for Finite-Sum Optimization Chaobing Song Yong Jiang Yi Ma 53 23 0 18 Jun 2020
Faster On-Device Training Using New Federated Momentum Algorithm Zhouyuan Huo Qian Yang Bin Gu Heng-Chiao Huang FedML 19 47 0 06 Feb 2020
Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks Zhaoxian Wu Qing Ling Tianyi Chen G. Giannakis FedML AAML 32 181 0 29 Dec 2019
Aggregated Gradient Langevin Dynamics Chao Zhang Jiahao Xie Zebang Shen P. Zhao Tengfei Zhou Hui Qian 28 1 0 21 Oct 2019
DEAM: Adaptive Momentum with Discriminative Weight for Stochastic Optimization Jiyang Bai Yuxiang Ren Jiawei Zhang ODL 21 1 0 25 Jul 2019
A Unifying Framework for Variance Reduction Algorithms for Finding Zeroes of Monotone Operators Xun Zhang W. Haskell Z. Ye 4 3 0 22 Jun 2019
Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources Yanghua Peng Hang Zhang Yifei Ma Tong He Zhi-Li Zhang Sheng Zha Mu Li 17 23 0 26 Apr 2019
Gradient Scheduling with Global Momentum for Non-IID Data Distributed Asynchronous Training Chengjie Li Ruixuan Li Yining Qi Yuhua Li Pan Zhou Song Guo Keqin Li 17 15 0 21 Feb 2019
A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates Kaiwen Zhou Fanhua Shang James Cheng 14 74 0 28 Jun 2018
Double Quantization for Communication-Efficient Distributed Optimization Yue Yu Jiaxiang Wu Longbo Huang MQ 19 57 0 25 May 2018
Backpropagation with N-D Vector-Valued Neurons Using Arbitrary Bilinear Products Zhe-Cheng Fan T. Chan Yi-Hsuan Yang J. Jang 23 7 0 24 May 2018
Taming Convergence for Asynchronous Stochastic Gradient Descent with Unbounded Delay in Non-Convex Learning Xin Zhang Jia-Wei Liu Zhengyuan Zhu 8 17 0 24 May 2018
Improved asynchronous parallel optimization analysis for stochastic incremental methods Rémi Leblond Fabian Pedregosa Simon Lacoste-Julien 16 70 0 11 Jan 2018
Gradient Sparsification for Communication-Efficient Distributed Optimization Jianqiao Wangni Jialei Wang Ji Liu Tong Zhang 15 522 0 26 Oct 2017
GIANT: Globally Improved Approximate Newton Method for Distributed Optimization Shusen Wang Farbod Roosta-Khorasani Peng Xu Michael W. Mahoney 28 127 0 11 Sep 2017
Variance-Reduced Stochastic Learning under Random Reshuffling Bicheng Ying Kun Yuan Ali H. Sayed 23 13 0 04 Aug 2017
Asynchronous Stochastic Block Coordinate Descent with Variance Reduction Bin Gu Zhouyuan Huo Heng-Chiao Huang 23 10 0 29 Oct 2016
Big Batch SGD: Automated Inference using Adaptive Batch Sizes Soham De A. Yadav David Jacobs Tom Goldstein ODL 14 62 0 18 Oct 2016
Federated Optimization: Distributed Machine Learning for On-Device Intelligence Jakub Konecný H. B. McMahan Daniel Ramage Peter Richtárik FedML 27 1,876 0 08 Oct 2016
Less than a Single Pass: Stochastically Controlled Stochastic Gradient Method Lihua Lei Michael I. Jordan 21 96 0 12 Sep 2016
AIDE: Fast and Communication Efficient Distributed Optimization Sashank J. Reddi Jakub Konecný Peter Richtárik Barnabás Póczós Alex Smola 11 150 0 24 Aug 2016
ASAGA: Asynchronous Parallel SAGA Rémi Leblond Fabian Pedregosa Simon Lacoste-Julien AI4TS 19 101 0 15 Jun 2016
Level Up Your Strategy: Towards a Descriptive Framework for Meaningful Enterprise Gamification Xinghao Pan 21 62 0 29 May 2016
Fast Stochastic Methods for Nonsmooth Nonconvex Optimization Sashank J. Reddi S. Sra Barnabás Póczós Alex Smola 15 54 0 23 May 2016
Optimal Margin Distribution Machine Teng Zhang Zhi-Hua Zhou 20 72 0 12 Apr 2016
Trading-off variance and complexity in stochastic gradient descent Vatsal Shah Megasthenis Asteris Anastasios Kyrillidis Sujay Sanghavi 17 13 0 22 Mar 2016
Fast Incremental Method for Nonconvex Optimization Sashank J. Reddi S. Sra Barnabás Póczós Alex Smola 10 44 0 19 Mar 2016
Accelerating Deep Neural Network Training with Inconsistent Stochastic Gradient Descent Linnan Wang Yi Yang Martin Renqiang Min S. Chakradhar 13 91 0 17 Mar 2016
SCOPE: Scalable Composite Optimization for Learning on Spark Shen-Yi Zhao Ru Xiang Yinghuan Shi Peng Gao Wu-Jun Li 16 16 0 30 Jan 2016
A Proximal Stochastic Gradient Method with Progressive Variance Reduction Lin Xiao Tong Zhang ODL 84 736 0 19 Mar 2014
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 177 683 0 07 Dec 2010