SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator

4 July 2018

Cong Fang

C. J. Li

Zhouchen Lin

Tong Zhang

ArXiv PDF HTML

Papers citing "SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator"

50 / 105 papers shown

Title
Variance Reduction Methods Do Not Need to Compute Full Gradients: Improved Efficiency through Shuffling Daniil Medyakov Gleb Molodtsov S. Chezhegov Alexey Rebrikov Aleksandr Beznosikov 103 0 0 21 Feb 2025
A single-loop SPIDER-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth optimization Wei Liu Yangyang Xu 71 3 0 31 Jan 2025
Adaptive Batch Size for Privately Finding Second-Order Stationary Points Daogao Liu Kunal Talwar 135 0 0 10 Oct 2024
Extended convexity and smoothness and their applications in deep learning Binchuan Qi Wei Gong Li Li 61 0 0 08 Oct 2024
Obtaining Lower Query Complexities through Lightweight Zeroth-Order Proximal Gradient Algorithms Bin Gu Xiyuan Wei Hualin Zhang Yi Chang Heng-Chiao Huang FedML 23 0 0 03 Oct 2024
An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness Xiaochuan Gong Jie Hao Mingrui Liu 46 2 0 28 Sep 2024
Gradient-Free Method for Heavily Constrained Nonconvex Optimization Wanli Shi Hongchang Gao Bin Gu 21 5 0 31 Aug 2024
Private Zeroth-Order Nonsmooth Nonconvex Optimization Qinzi Zhang Hoang Tran Ashok Cutkosky 40 4 0 27 Jun 2024
Comparisons Are All You Need for Optimizing Smooth Functions Chenyi Zhang Tongyang Li AAML 37 1 0 19 May 2024
Robust Decentralized Learning with Local Updates and Gradient Tracking Sajjad Ghiasvand Amirhossein Reisizadeh Mahnoosh Alizadeh Ramtin Pedarsani 42 3 0 02 May 2024
Stochastic Halpern iteration in normed spaces and applications to reinforcement learning Mario Bravo Juan Pablo Contreras 45 3 0 19 Mar 2024
Non-Convex Stochastic Composite Optimization with Polyak Momentum Yuan Gao Anton Rodomanov Sebastian U. Stich 37 6 0 05 Mar 2024
Probabilistic Guarantees of Stochastic Recursive Gradient in Non-Convex Finite Sum Problems Yanjie Zhong Jiaqi Li Soumendra Lahiri 27 1 0 29 Jan 2024
On the Stochastic (Variance-Reduced) Proximal Gradient Method for Regularized Expected Reward Optimization Ling Liang Haizhao Yang 14 1 0 23 Jan 2024
A Coefficient Makes SVRG Effective Yida Yin Zhiqiu Xu Zhiyuan Li Trevor Darrell Zhuang Liu 33 1 0 09 Nov 2023
Adaptive Mirror Descent Bilevel Optimization Feihu Huang 40 1 0 08 Nov 2023
Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization Zhen Qin Zhishuai Liu Pan Xu 20 1 0 24 Oct 2023
Decentralized Gradient-Free Methods for Stochastic Non-Smooth Non-Convex Optimization Zhenwei Lin Jingfan Xia Qi Deng Luo Luo 31 3 0 18 Oct 2023
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates Ahmad Rammal Kaja Gruntkowska Nikita Fedin Eduard A. Gorbunov Peter Richtárik 45 5 0 15 Oct 2023
On the Parallel Complexity of Multilevel Monte Carlo in Stochastic Gradient Descent Kei Ishikawa BDL 63 0 0 03 Oct 2023
Oracle Complexity Reduction for Model-free LQR: A Stochastic Variance-Reduced Policy Gradient Approach Leonardo F. Toso Han Wang James Anderson 37 2 0 19 Sep 2023
Achieving Linear Speedup in Decentralized Stochastic Compositional Minimax Optimization Hongchang Gao 37 1 0 25 Jul 2023
Variance-reduced accelerated methods for decentralized stochastic double-regularized nonconvex strongly-concave minimax problems Gabriel Mancino-Ball Yangyang Xu 20 8 0 14 Jul 2023
Efficient preconditioned stochastic gradient descent for estimation in latent variable models C. Baey Maud Delattre E. Kuhn Jean-Benoist Léger Sarah Lemler 21 4 0 22 Jun 2023
Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation Davoud Ataee Tarzanagh Mingchen Li Pranay Sharma Samet Oymak 26 0 0 02 Jun 2023
How to escape sharp minima with random perturbations Kwangjun Ahn Ali Jadbabaie S. Sra ODL 32 6 0 25 May 2023
Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Features Aleksandr Beznosikov David Dobre Gauthier Gidel 25 5 0 23 Apr 2023
On Momentum-Based Gradient Methods for Bilevel Optimization with Nonconvex Lower-Level Feihu Huang 32 18 0 07 Mar 2023
Private (Stochastic) Non-Convex Optimization Revisited: Second-Order Stationary Points and Excess Risks Arun Ganesh Daogao Liu Sewoong Oh Abhradeep Thakurta ODL 27 12 0 20 Feb 2023
Solving stochastic weak Minty variational inequalities without increasing batch size Thomas Pethick Olivier Fercoq P. Latafat Panagiotis Patrinos V. Cevher 18 23 0 17 Feb 2023
Breaking the Lower Bound with (Little) Structure: Acceleration in Non-Convex Stochastic Optimization with Heavy-Tailed Noise Zijian Liu Jiawei Zhang Zhengyuan Zhou 32 12 0 14 Feb 2023
Stochastic Dimension-reduced Second-order Methods for Policy Optimization Jinsong Liu Chen Xie Qinwen Deng Dongdong Ge Yi-Li Ye 29 1 0 28 Jan 2023
Faster Gradient-Free Algorithms for Nonsmooth Nonconvex Stochastic Optimization Le‐Yu Chen Jing Xu Luo Luo 31 15 0 16 Jan 2023
Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction Bowen Lei Dongkuan Xu Ruqi Zhang Shuren He Bani Mallick 32 6 0 09 Jan 2023
Stochastic Variable Metric Proximal Gradient with variance reduction for non-convex composite optimization G. Fort Eric Moulines 46 6 0 02 Jan 2023
Stochastic Inexact Augmented Lagrangian Method for Nonconvex Expectation Constrained Optimization Zichong Li Pinzhuo Chen Sijia Liu Songtao Lu Yangyang Xu 27 17 0 19 Dec 2022
Variance-Reduced Conservative Policy Iteration Naman Agarwal Brian Bullins Karan Singh 29 3 0 12 Dec 2022
Decentralized Stochastic Gradient Descent Ascent for Finite-Sum Minimax Problems Hongchang Gao 24 16 0 06 Dec 2022
An Efficient Stochastic Algorithm for Decentralized Nonconvex-Strongly-Concave Minimax Optimization Le‐Yu Chen Haishan Ye Luo Luo 65 5 0 05 Dec 2022
Zeroth-Order Alternating Gradient Descent Ascent Algorithms for a Class of Nonconvex-Nonconcave Minimax Problems Zi Xu Ziqi Wang Junlin Wang Y. Dai 18 11 0 24 Nov 2022
Adaptive Stochastic Optimisation of Nonconvex Composite Objectives Weijia Shao F. Sivrikaya S. Albayrak 16 0 0 21 Nov 2022
An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methods Yanli Liu Kaipeng Zhang Tamer Basar W. Yin 42 102 0 15 Nov 2022
Momentum Aggregation for Private Non-convex ERM Hoang Tran Ashok Cutkosky 26 14 0 12 Oct 2022
SARAH-based Variance-reduced Algorithm for Stochastic Finite-sum Cocoercive Variational Inequalities Aleksandr Beznosikov Alexander Gasnikov 37 2 0 12 Oct 2022
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients Hualin Zhang Huan Xiong Bin Gu 32 7 0 04 Oct 2022
On Quantum Speedups for Nonconvex Optimization via Quantum Tunneling Walks Yizhou Liu Weijie J. Su Tongyang Li 24 17 0 29 Sep 2022
NET-FLEET: Achieving Linear Convergence Speedup for Fully Decentralized Federated Learning with Heterogeneous Data Xin Zhang Minghong Fang Zhuqing Liu Haibo Yang Jia-Wei Liu Zhengyuan Zhu FedML 22 14 0 17 Aug 2022
SYNTHESIS: A Semi-Asynchronous Path-Integrated Stochastic Gradient Method for Distributed Learning in Computing Clusters Zhuqing Liu Xin Zhang Jia-Wei Liu 32 1 0 17 Aug 2022
Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization Wei Jiang Gang Li Yibo Wang Lijun Zhang Tianbao Yang 35 16 0 18 Jul 2022
SPIRAL: A superlinearly convergent incremental proximal algorithm for nonconvex finite sum minimization Pourya Behmandpoor P. Latafat Andreas Themelis Marc Moonen Panagiotis Patrinos 29 2 0 17 Jul 2022