Lower Bounds for Non-Convex Stochastic Optimization

5 December 2019

Papers citing "Lower Bounds for Non-Convex Stochastic Optimization"

50 / 88 papers shown

Title
Observability conditions for neural state-space models with eigenvalues and their roots of unity Andrew Gracyk 209 0 0 22 Apr 2025
Nested Stochastic Gradient Descent for (Generalized) Sinkhorn Distance-Regularized Distributionally Robust Optimization Yue Yang Yi Zhou Zhaosong Lu 49 0 0 29 Mar 2025
Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization Dmitry Kovalev 59 1 0 16 Mar 2025
Variance Reduction Methods Do Not Need to Compute Full Gradients: Improved Efficiency through Shuffling Daniil Medyakov Gleb Molodtsov S. Chezhegov Alexey Rebrikov Aleksandr Beznosikov 107 0 0 21 Feb 2025
Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity Artavazd Maranjyan A. Tyurin Peter Richtárik 49 3 0 27 Jan 2025
A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness Xiaochuan Gong Jie Hao Mingrui Liu 57 3 0 31 Dec 2024
From Gradient Clipping to Normalization for Heavy Tailed SGD Florian Hübler Ilyas Fatkhullin Niao He 42 5 0 17 Oct 2024
Adaptive Batch Size for Privately Finding Second-Order Stationary Points Daogao Liu Kunal Talwar 204 0 0 10 Oct 2024
Extended convexity and smoothness and their applications in deep learning Binchuan Qi Wei Gong Li Li 63 0 0 08 Oct 2024
An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness Xiaochuan Gong Jie Hao Mingrui Liu 52 2 0 28 Sep 2024
Convergence Conditions for Stochastic Line Search Based Optimization of Over-parametrized Models Matteo Lapucci Davide Pucci 37 1 0 06 Aug 2024
Private Zeroth-Order Nonsmooth Nonconvex Optimization Qinzi Zhang Hoang Tran Ashok Cutkosky 42 4 0 27 Jun 2024
Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum H. Cai Sulaiman A. Alghunaim Ali H.Sayed 52 1 0 18 Jun 2024
Random Scaling and Momentum for Non-smooth Non-convex Optimization Qinzi Zhang Ashok Cutkosky 43 4 0 16 May 2024
Learning Optimal Deterministic Policies with Stochastic Policy Gradients Alessandro Montenegro Marco Mussi Alberto Maria Metelli Matteo Papini 48 2 0 03 May 2024
Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation Aaron Mishkin Mert Pilanci Mark Schmidt 66 1 0 03 Apr 2024
Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance Qi Zhang Yi Zhou Shaofeng Zou 42 4 0 01 Apr 2024
Stochastic Halpern iteration in normed spaces and applications to reinforcement learning Mario Bravo Juan Pablo Contreras 48 3 0 19 Mar 2024
Non-Convex Stochastic Composite Optimization with Polyak Momentum Yuan Gao Anton Rodomanov Sebastian U. Stich 39 6 0 05 Mar 2024
Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad Sayantan Choudhury N. Tupitsa Nicolas Loizou Samuel Horváth Martin Takáč Eduard A. Gorbunov 40 1 0 05 Mar 2024
Tuning-Free Stochastic Optimization Ahmed Khaled Chi Jin 32 7 0 12 Feb 2024
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions Yusu Hong Junhong Lin 48 12 0 06 Feb 2024
How Free is Parameter-Free Stochastic Optimization? Amit Attia Tomer Koren ODL 47 5 0 05 Feb 2024
Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis Jie Hao Xiaochuan Gong Mingrui Liu 33 7 0 17 Jan 2024
Convergence Rates for Stochastic Approximation: Biased Noise with Unbounded Variance, and Applications Rajeeva Laxman Karandikar M. Vidyasagar 27 8 0 05 Dec 2023
Demystifying the Myths and Legends of Nonconvex Convergence of SGD Aritra Dutta El Houcine Bergou Soumia Boucherouite Nicklas Werge M. Kandemir Xin Li 28 0 0 19 Oct 2023
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates Ahmad Rammal Kaja Gruntkowska Nikita Fedin Eduard A. Gorbunov Peter Richtárik 47 5 0 15 Oct 2023
Variance-reduced accelerated methods for decentralized stochastic double-regularized nonconvex strongly-concave minimax problems Gabriel Mancino-Ball Yangyang Xu 20 8 0 14 Jul 2023
Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning Kuan-Fu Ding Jingyang Li Kim-Chuan Toh 33 8 0 26 Jun 2023
Symmetry & Critical Points for Symmetric Tensor Decomposition Problems Yossi Arjevani Gal Vinograd 37 5 0 13 Jun 2023
Convex and Non-convex Optimization Under Generalized Smoothness Haochuan Li Jian Qian Yi Tian Alexander Rakhlin Ali Jadbabaie 13 35 0 02 Jun 2023
How to escape sharp minima with random perturbations Kwangjun Ahn Ali Jadbabaie S. Sra ODL 34 6 0 25 May 2023
On the Optimal Batch Size for Byzantine-Robust Distributed Learning Yi-Rui Yang Chang-Wei Shi Wu-Jun Li FedML AAML 27 0 0 23 May 2023
Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods Junchi Yang Xiang Li Ilyas Fatkhullin Niao He 42 15 0 21 May 2023
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression Yutong He Xinmeng Huang Yiming Chen W. Yin Kun Yuan 36 7 0 12 May 2023
Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold S. Schechtman D. Tiapkin Michael Muehlebach Eric Moulines 27 7 0 16 Mar 2023
Enhanced Adaptive Gradient Algorithms for Nonconvex-PL Minimax Optimization Feihu Huang Chunyu Xuan Xinrui Wang Siqi Zhang Songcan Chen 28 7 0 07 Mar 2023
A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization Tesi Xiao Xuxing Chen Krishnakumar Balasubramanian Saeed Ghadimi 36 10 0 20 Feb 2023
Solving stochastic weak Minty variational inequalities without increasing batch size Thomas Pethick Olivier Fercoq P. Latafat Panagiotis Patrinos V. Cevher 24 23 0 17 Feb 2023
SGD with AdaGrad Stepsizes: Full Adaptivity with High Probability to Unknown Parameters, Unbounded Gradients and Affine Variance Amit Attia Tomer Koren ODL 22 26 0 17 Feb 2023
Deterministic Nonsmooth Nonconvex Optimization Michael I. Jordan Guy Kornowski Tianyi Lin Ohad Shamir Manolis Zampetakis 57 26 0 16 Feb 2023
Breaking the Lower Bound with (Little) Structure: Acceleration in Non-Convex Stochastic Optimization with Heavy-Tailed Noise Zijian Liu Jiawei Zhang Zhengyuan Zhou 34 12 0 14 Feb 2023
Stochastic Inexact Augmented Lagrangian Method for Nonconvex Expectation Constrained Optimization Zichong Li Pinzhuo Chen Sijia Liu Songtao Lu Yangyang Xu 35 17 0 19 Dec 2022
Variance-Reduced Conservative Policy Iteration Naman Agarwal Brian Bullins Karan Singh 32 3 0 12 Dec 2022
An Efficient Stochastic Algorithm for Decentralized Nonconvex-Strongly-Concave Minimax Optimization Le‐Yu Chen Haishan Ye Luo Luo 73 5 0 05 Dec 2022
Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization Ali Kavis Stratis Skoulakis Kimon Antonakopoulos L. Dadi V. Cevher 27 15 0 03 Nov 2022
Momentum Aggregation for Private Non-convex ERM Hoang Tran Ashok Cutkosky 28 14 0 12 Oct 2022
Annihilation of Spurious Minima in Two-Layer ReLU Networks Yossi Arjevani M. Field 16 8 0 12 Oct 2022
Stochastic Constrained DRO with a Complexity Independent of Sample Size Q. Qi Jiameng Lyu Kung-Sik Chan E. Bai Tianbao Yang 50 15 0 11 Oct 2022
Taming Fat-Tailed ("Heavier-Tailed'' with Potentially Infinite Variance) Noise in Federated Learning Haibo Yang Pei-Yuan Qiu Jia Liu FedML 37 12 0 03 Oct 2022