PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimization

25 August 2020

Papers citing "PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimization"

39 / 39 papers shown

Title
An Enhanced Zeroth-Order Stochastic Frank-Wolfe Framework for Constrained Finite-Sum Optimization Haishan Ye Yinghui Huang Hao Di Xiangyu Chang 87 0 0 13 Jan 2025
Stochastic Halpern iteration in normed spaces and applications to reinforcement learning Mario Bravo Juan Pablo Contreras 63 3 0 19 Mar 2024
An Inexact Halpern Iteration with Application to Distributionally Robust Optimization Ling Liang Zusen Xu Kim-Chuan Toh Jia Jie Zhu 86 4 0 08 Feb 2024
EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback Peter Richtárik Igor Sokolov Ilyas Fatkhullin 51 142 0 09 Jun 2021
ANITA: An Optimal Loopless Accelerated Variance-Reduced Gradient Method Zhize Li 55 14 0 21 Mar 2021
ZeroSARAH: Efficient Nonconvex Finite-Sum Optimization with Zero Full Gradient Computation Zhize Li Slavomír Hanzely Peter Richtárik 20 31 0 02 Mar 2021
MARINA: Faster Non-Convex Distributed Learning with Compression Eduard A. Gorbunov Konstantin Burlachenko Zhize Li Peter Richtárik 55 109 0 15 Feb 2021
A Unified Analysis of Stochastic Gradient Methods for Nonconvex Federated Optimization Zhize Li Peter Richtárik FedML 60 36 0 12 Jun 2020
Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization Zhize Li D. Kovalev Xun Qian Peter Richtárik FedML AI4CE 78 136 0 26 Feb 2020
Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization Samuel Horváth Lihua Lei Peter Richtárik Michael I. Jordan 90 30 0 13 Feb 2020
Better Theory for SGD in the Nonconvex World Ahmed Khaled Peter Richtárik 38 182 0 09 Feb 2020
Lower Bounds for Non-Convex Stochastic Optimization Yossi Arjevani Y. Carmon John C. Duchi Dylan J. Foster Nathan Srebro Blake E. Woodworth 67 349 0 05 Dec 2019
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 280 42,038 0 03 Dec 2019
A General Analysis Framework of Lower Complexity Bounds for Finite-Sum Optimization Guangzeng Xie Luo Luo Zhihua Zhang 32 4 0 22 Aug 2019
On the Convergence of SARAH and Beyond Bingcong Li Meng Ma G. Giannakis 48 26 0 05 Jun 2019
A unified variance-reduced accelerated gradient method for convex optimization Guanghui Lan Zhize Li Yi Zhou 39 61 0 29 May 2019
Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization Rong Ge Zhize Li Weiyao Wang Xiang Wang 42 34 0 01 May 2019
SSRGD: Simple Stochastic Recursive Gradient Descent for Escaping Saddle Points Zhize Li 42 38 0 19 Apr 2019
ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization Nhan H. Pham Lam M. Nguyen Dzung Phan Quoc Tran-Dinh 33 140 0 15 Feb 2019
Lower Bounds for Smooth Nonconvex Finite-Sum Optimization Dongruo Zhou Quanquan Gu 38 45 0 31 Jan 2019
Don't Jump Through Hoops and Remove Those Loops: SVRG and Katyusha are Better Without the Outer Loop D. Kovalev Samuel Horváth Peter Richtárik 65 156 0 24 Jan 2019
SpiderBoost and Momentum: Faster Stochastic Variance Reduction Algorithms Zhe Wang Kaiyi Ji Yi Zhou Yingbin Liang Vahid Tarokh ODL 59 81 0 25 Oct 2018
A Fast Anderson-Chebyshev Acceleration for Nonlinear Optimization Zhize Li Jian Li 37 19 0 07 Sep 2018
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator Cong Fang C. J. Li Zhouchen Lin Tong Zhang 81 572 0 04 Jul 2018
Stochastic Nested Variance Reduction for Nonconvex Optimization Dongruo Zhou Pan Xu Quanquan Gu 49 147 0 20 Jun 2018
A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimization Zhize Li Jian Li 58 116 0 13 Feb 2018
Non-convex Optimization for Machine Learning Prateek Jain Purushottam Kar 107 480 0 21 Dec 2017
Random gradient extrapolation for distributed and stochastic optimization Guanghui Lan Yi Zhou 42 52 0 15 Nov 2017
Natasha 2: Faster Non-Convex Optimization Than SGD Zeyuan Allen-Zhu ODL 64 245 0 29 Aug 2017
SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient Lam M. Nguyen Jie Liu K. Scheinberg Martin Takáč ODL 146 601 0 01 Mar 2017
Tight Complexity Bounds for Optimizing Composite Objectives Blake E. Woodworth Nathan Srebro 109 185 0 25 May 2016
Stochastic Variance Reduction for Nonconvex Optimization Sashank J. Reddi Ahmed S. Hefny S. Sra Barnabás Póczós Alex Smola 80 598 0 19 Mar 2016
Katyusha: The First Direct Acceleration of Stochastic Gradient Methods Zeyuan Allen-Zhu ODL 85 580 0 18 Mar 2016
Variance Reduction for Faster Non-Convex Optimization Zeyuan Allen-Zhu Elad Hazan ODL 100 391 0 17 Mar 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.4K 192,638 0 10 Dec 2015
An optimal randomized incremental gradient method Guanghui Lan Yi Zhou 101 220 0 08 Jul 2015
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 957 99,991 0 04 Sep 2014
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives Aaron Defazio Francis R. Bach Simon Lacoste-Julien ODL 105 1,817 0 01 Jul 2014
Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming Saeed Ghadimi Guanghui Lan ODL 84 1,538 0 22 Sep 2013