SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator

4 July 2018

Cong Fang

C. J. Li

Zhouchen Lin

Tong Zhang

ArXiv PDF HTML

Papers citing "SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator"

50 / 109 papers shown

Title
Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization Wei Jiang Gang Li Yibo Wang Lijun Zhang Tianbao Yang 35 16 0 18 Jul 2022
SPIRAL: A superlinearly convergent incremental proximal algorithm for nonconvex finite sum minimization Pourya Behmandpoor P. Latafat Andreas Themelis Marc Moonen Panagiotis Patrinos 29 2 0 17 Jul 2022
Quantum Neural Network Compression Zhirui Hu Peiyan Dong Zhepeng Wang Youzuo Lin Yanzhi Wang Weiwen Jiang GNN 27 28 0 04 Jul 2022
On the Convergence of Momentum-Based Algorithms for Federated Bilevel Optimization Problems Hongchang Gao FedML 28 1 0 28 Apr 2022
AUC Maximization in the Era of Big Data and AI: A Survey Tianbao Yang Yiming Ying 38 179 0 28 Mar 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 28 8 0 18 Feb 2022
Optimal Algorithms for Stochastic Multi-Level Compositional Optimization Wei Jiang Bokun Wang Yibo Wang Lijun Zhang Tianbao Yang 79 17 0 15 Feb 2022
Decentralized Stochastic Variance Reduced Extragradient Method Luo Luo Haishan Ye 27 7 0 01 Feb 2022
L-SVRG and L-Katyusha with Adaptive Sampling Boxin Zhao Boxiang Lyu Mladen Kolar 21 3 0 31 Jan 2022
A Novel Convergence Analysis for Algorithms of the Adam Family Zhishuai Guo Yi Tian Xu W. Yin R. L. Jin Tianbao Yang 39 47 0 07 Dec 2021
Escape saddle points by a simple gradient-descent based algorithm Chenyi Zhang Tongyang Li ODL 31 15 0 28 Nov 2021
Random-reshuffled SARAH does not need a full gradient computations Aleksandr Beznosikov Martin Takáč 26 7 0 26 Nov 2021
Federated Expectation Maximization with heterogeneity mitigation and variance reduction Aymeric Dieuleveut G. Fort Eric Moulines Geneviève Robin FedML 31 5 0 03 Nov 2021
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 40 1 0 25 Oct 2021
On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learning Weichao Mao Lin F. Yang Kaipeng Zhang Tamer Bacsar 39 57 0 12 Oct 2021
The Number of Steps Needed for Nonconvex Optimization of a Deep Learning Optimizer is a Rational Function of Batch Size Hideaki Iiduka 26 1 0 26 Aug 2021
Enhanced Bilevel Optimization via Bregman Distance Feihu Huang Junyi Li Shangqian Gao Heng-Chiao Huang 27 33 0 26 Jul 2021
A general sample complexity analysis of vanilla policy gradient Rui Yuan Robert Mansel Gower A. Lazaric 79 62 0 23 Jul 2021
Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent Spencer Frei Quanquan Gu 26 25 0 25 Jun 2021
Provably Faster Algorithms for Bilevel Optimization Junjie Yang Kaiyi Ji Yingbin Liang 49 132 0 08 Jun 2021
Escaping Saddle Points with Compressed SGD Dmitrii Avdiukhin G. Yaroslavtsev 19 4 0 21 May 2021
Complexity Lower Bounds for Nonconvex-Strongly-Concave Min-Max Optimization Haochuan Li Yi Tian Jingzhao Zhang Ali Jadbabaie 24 40 0 18 Apr 2021
Greedy-GQ with Variance Reduction: Finite-time Analysis and Improved Complexity Shaocong Ma Ziyi Chen Yi Zhou Shaofeng Zou 17 11 0 30 Mar 2021
The Complexity of Nonconvex-Strongly-Concave Minimax Optimization Siqi Zhang Junchi Yang Cristóbal Guzmán Negar Kiyavash Niao He 33 61 0 29 Mar 2021
ANITA: An Optimal Loopless Accelerated Variance-Reduced Gradient Method Zhize Li 43 14 0 21 Mar 2021
$Private Stochastic Convex Optimization: Optimal Rates in $\ell_1$ Geometry$ Private Stochastic Convex Optimization: Optimal Rates in $\ell_1$ Geometry Hilal Asi Vitaly Feldman Tomer Koren Kunal Talwar 17 91 0 02 Mar 2021
Non-Euclidean Differentially Private Stochastic Convex Optimization: Optimal Rates in Linear Time Raef Bassily Cristóbal Guzmán Anupama Nandi 54 66 0 01 Mar 2021
MARINA: Faster Non-Convex Distributed Learning with Compression Eduard A. Gorbunov Konstantin Burlachenko Zhize Li Peter Richtárik 39 109 0 15 Feb 2021
Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training Cong Fang Hangfeng He Qi Long Weijie J. Su FAtt 130 167 0 29 Jan 2021
Efficient Semi-Implicit Variational Inference Vincent Moens Hang Ren A. Maraval Rasul Tutunov Jun Wang H. Ammar 85 6 0 15 Jan 2021
Faster Non-Convex Federated Learning via Global and Local Momentum Rudrajit Das Anish Acharya Abolfazl Hashemi Sujay Sanghavi Inderjit S. Dhillon Ufuk Topcu FedML 37 82 0 07 Dec 2020
A Stochastic Path-Integrated Differential EstimatoR Expectation Maximization Algorithm G. Fort Eric Moulines Hoi-To Wai TPM 14 7 0 30 Nov 2020
Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters Filip Hanzely 34 0 0 26 Aug 2020
PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimization Zhize Li Hongyan Bao Xiangliang Zhang Peter Richtárik ODL 31 126 0 25 Aug 2020
Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization Tianyi Chen Yuejiao Sun W. Yin 48 81 0 25 Aug 2020
Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning Bin Gu An Xu Zhouyuan Huo Cheng Deng Heng-Chiao Huang FedML 38 27 0 14 Aug 2020
Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations Yossi Arjevani Y. Carmon John C. Duchi Dylan J. Foster Ayush Sekhari Karthik Sridharan 84 53 0 24 Jun 2020
An Online Method for A Class of Distributionally Robust Optimization with Non-Convex Objectives Qi Qi Zhishuai Guo Yi Tian Xu R. L. Jin Tianbao Yang 33 44 0 17 Jun 2020
Optimal Complexity in Decentralized Training Yucheng Lu Christopher De Sa 38 72 0 15 Jun 2020
An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Bias Lu Yu Krishnakumar Balasubramanian S. Volgushev Murat A. Erdogdu 35 50 0 14 Jun 2020
Biased Stochastic First-Order Methods for Conditional Stochastic Optimization and Applications in Meta Learning Yifan Hu Siqi Zhang Xin Chen Niao He ODL 41 55 0 25 Feb 2020
Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization Samuel Horváth Lihua Lei Peter Richtárik Michael I. Jordan 57 30 0 13 Feb 2020
Variance Reduced Coordinate Descent with Acceleration: New Method With a Surprising Application to Finite-Sum Problems Filip Hanzely D. Kovalev Peter Richtárik 35 17 0 11 Feb 2020
Momentum Improves Normalized SGD Ashok Cutkosky Harsh Mehta ODL 10 118 0 09 Feb 2020
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 22 168 0 19 Dec 2019
History-Gradient Aided Batch Size Adaptation for Variance Reduced Algorithms Kaiyi Ji Zhe Wang Bowen Weng Yi Zhou Wei Zhang Yingbin Liang ODL 18 5 0 21 Oct 2019
Sample Efficient Policy Gradient Methods with Recursive Variance Reduction Pan Xu F. Gao Quanquan Gu 28 83 0 18 Sep 2019
Second-Order Guarantees of Stochastic Gradient Descent in Non-Convex Optimization Stefan Vlaski Ali H. Sayed ODL 26 21 0 19 Aug 2019
Stochastic First-order Methods for Convex and Nonconvex Functional Constrained Optimization Digvijay Boob Qi Deng Guanghui Lan 44 93 0 07 Aug 2019
Distributed Learning in Non-Convex Environments -- Part II: Polynomial Escape from Saddle-Points Stefan Vlaski Ali H. Sayed 21 53 0 03 Jul 2019