Momentum-Based Variance Reduction in Non-Convex SGD

24 May 2019

Papers citing "Momentum-Based Variance Reduction in Non-Convex SGD"

46 / 96 papers shown

Title
Stochastic Constrained DRO with a Complexity Independent of Sample Size Q. Qi Jiameng Lyu Kung-Sik Chan E. Bai Tianbao Yang 50 15 0 11 Oct 2022
SENDER: SEmi-Nonlinear Deep Efficient Reconstructor for Extraction Canonical, Meta, and Sub Functional Connectivity in the Human Brain Wei Zhang Yun-Jian Bao 11 0 0 12 Sep 2022
Robustness to Unbounded Smoothness of Generalized SignSGD M. Crawshaw Mingrui Liu Francesco Orabona Wei Zhang Zhenxun Zhuang AAML 36 66 0 23 Aug 2022
Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization Wei Jiang Gang Li Yibo Wang Lijun Zhang Tianbao Yang 35 16 0 18 Jul 2022
Communication-Efficient Federated Learning With Data and Client Heterogeneity Hossein Zakerinia Shayan Talaei Giorgi Nadiradze Dan Alistarh FedML 34 7 0 20 Jun 2022
Multi-block Min-max Bilevel Optimization with Applications in Multi-task Deep AUC Maximization Quanqi Hu Yongjian Zhong Tianbao Yang 38 16 0 01 Jun 2022
SADAM: Stochastic Adam, A Stochastic Operator for First-Order Gradient-based Optimizer Wei Zhang Yun-Jian Bao ODL 33 2 0 20 May 2022
Local Stochastic Bilevel Optimization with Momentum-Based Variance Reduction Junyi Li Feihu Huang Heng-Chiao Huang FedML 28 27 0 03 May 2022
On the Convergence of Momentum-Based Algorithms for Federated Bilevel Optimization Problems Hongchang Gao FedML 49 1 0 28 Apr 2022
AUC Maximization in the Era of Big Data and AI: A Survey Tianbao Yang Yiming Ying 44 179 0 28 Mar 2022
A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization With Non-isolated Local Minima Tae-Eon Ko Xiantao Li 30 2 0 21 Mar 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 31 8 0 18 Feb 2022
Optimal Algorithms for Stochastic Multi-Level Compositional Optimization Wei Jiang Bokun Wang Yibo Wang Lijun Zhang Tianbao Yang 81 17 0 15 Feb 2022
PAGE-PG: A Simple and Loopless Variance-Reduced Policy Gradient Method with Probabilistic Gradient Estimation Matilde Gargiani Andrea Zanelli Andrea Martinelli Tyler H. Summers John Lygeros 33 14 0 01 Feb 2022
A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse Junyi Li Bin Gu Heng-Chiao Huang 26 72 0 09 Dec 2021
A Novel Convergence Analysis for Algorithms of the Adam Family Zhishuai Guo Yi Tian Xu W. Yin Rong Jin Tianbao Yang 39 48 0 07 Dec 2021
Training Structured Neural Networks Through Manifold Identification and Variance Reduction Zih-Syuan Huang Ching-pei Lee AAML 50 9 0 05 Dec 2021
Random-reshuffled SARAH does not need a full gradient computations Aleksandr Beznosikov Martin Takáč 31 7 0 26 Nov 2021
Linear Speedup in Personalized Collaborative Learning El Mahdi Chayti Sai Praneeth Karimireddy Sebastian U. Stich Nicolas Flammarion Martin Jaggi FedML 21 13 0 10 Nov 2021
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 43 1 0 25 Oct 2021
On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learning Weichao Mao Lin F. Yang Kaipeng Zhang Tamer Bacsar 46 57 0 12 Oct 2021
Enhanced Bilevel Optimization via Bregman Distance Feihu Huang Junyi Li Shangqian Gao Heng-Chiao Huang 32 33 0 26 Jul 2021
A general sample complexity analysis of vanilla policy gradient Rui Yuan Robert Mansel Gower A. Lazaric 82 62 0 23 Jul 2021
Zeroth and First Order Stochastic Frank-Wolfe Algorithms for Constrained Optimization Zeeshan Akhtar K. Rajawat 37 6 0 14 Jul 2021
A Survey on Fault-tolerance in Distributed Optimization and Machine Learning Shuo Liu AI4CE OOD 58 13 0 16 Jun 2021
Provably Faster Algorithms for Bilevel Optimization Junjie Yang Kaiyi Ji Yingbin Liang 51 132 0 08 Jun 2021
On the Convergence Rate of Off-Policy Policy Optimization Methods with Density-Ratio Correction Jiawei Huang Nan Jiang 19 5 0 02 Jun 2021
Randomized Stochastic Variance-Reduced Methods for Multi-Task Stochastic Bilevel Optimization Zhishuai Guo Quan Hu Lijun Zhang Tianbao Yang 61 30 0 05 May 2021
GT-STORM: Taming Sample, Communication, and Memory Complexities in Decentralized Non-Convex Learning Xin Zhang Jia Liu Zhengyuan Zhu Elizabeth S. Bentley 49 14 0 04 May 2021
Doubly Robust Off-Policy Actor-Critic: Convergence and Optimality Tengyu Xu Zhuoran Yang Zhaoran Wang Yingbin Liang OffRL 47 24 0 23 Feb 2021
SVRG Meets AdaGrad: Painless Variance Reduction Benjamin Dubois-Taine Sharan Vaswani Reza Babanezhad Mark Schmidt Simon Lacoste-Julien 20 18 0 18 Feb 2021
A Near-Optimal Algorithm for Stochastic Bilevel Optimization via Double-Momentum Prashant Khanduri Siliang Zeng Mingyi Hong Hoi-To Wai Zhaoran Wang Zhuoran Yang 22 129 0 15 Feb 2021
A Hybrid Variance-Reduced Method for Decentralized Stochastic Non-Convex Optimization Ran Xin U. Khan S. Kar 27 39 0 12 Feb 2021
Learning from History for Byzantine Robust Optimization Sai Praneeth Karimireddy Lie He Martin Jaggi FedML AAML 30 174 0 18 Dec 2020
Faster Non-Convex Federated Learning via Global and Local Momentum Rudrajit Das Anish Acharya Abolfazl Hashemi Sujay Sanghavi Inderjit S. Dhillon Ufuk Topcu FedML 40 82 0 07 Dec 2020
Single-Timescale Stochastic Nonconvex-Concave Optimization for Smooth Nonlinear TD Learning Shuang Qiu Zhuoran Yang Xiaohan Wei Jieping Ye Zhaoran Wang 33 38 0 23 Aug 2020
Riemannian stochastic recursive momentum method for non-convex optimization Andi Han Junbin Gao ODL 28 17 0 11 Aug 2020
Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations Yossi Arjevani Y. Carmon John C. Duchi Dylan J. Foster Ayush Sekhari Karthik Sridharan 90 53 0 24 Jun 2020
An Online Method for A Class of Distributionally Robust Optimization with Non-Convex Objectives Qi Qi Zhishuai Guo Yi Tian Xu Rong Jin Tianbao Yang 33 44 0 17 Jun 2020
Momentum-based variance-reduced proximal stochastic gradient method for composite nonconvex stochastic optimization Yangyang Xu Yibo Xu 33 23 0 31 May 2020
Stochastic Recursive Momentum for Policy Gradient Methods Huizhuo Yuan Xiangru Lian Ji Liu Yuren Zhou 23 31 0 09 Mar 2020
A Unified Convergence Analysis for Shuffling-Type Gradient Methods Lam M. Nguyen Quoc Tran-Dinh Dzung Phan Phuong Ha Nguyen Marten van Dijk 39 78 0 19 Feb 2020
Momentum Improves Normalized SGD Ashok Cutkosky Harsh Mehta ODL 18 119 0 09 Feb 2020
Demon: Improved Neural Network Training with Momentum Decay John Chen Cameron R. Wolfe Zhaoqi Li Anastasios Kyrillidis ODL 24 15 0 11 Oct 2019
One Sample Stochastic Frank-Wolfe Mingrui Zhang Zebang Shen Aryan Mokhtari Hamed Hassani Amin Karbasi 14 57 0 10 Oct 2019
A Hybrid Stochastic Optimization Framework for Stochastic Composite Nonconvex Optimization Quoc Tran-Dinh Nhan H. Pham T. Dzung Lam M. Nguyen 27 49 0 08 Jul 2019