Beyond Uniform Smoothness: A Stopped Analysis of Adaptive SGD

Beyond Uniform Smoothness: A Stopped Analysis of Adaptive SGD

13 February 2023

Sanjay Shakkottai

Papers citing "Beyond Uniform Smoothness: A Stopped Analysis of Adaptive SGD"

12 / 12 papers shown

Title
Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization Dmitry Kovalev 57 1 0 16 Mar 2025
A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness Xiaochuan Gong Jie Hao Mingrui Liu 57 3 0 31 Dec 2024
An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness Xiaochuan Gong Jie Hao Mingrui Liu 52 2 0 28 Sep 2024
An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes Antonio Orvieto Lin Xiao 45 3 0 05 Jul 2024
Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance Qi Zhang Yi Zhou Shaofeng Zou 42 4 0 01 Apr 2024
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions Yusu Hong Junhong Lin 46 12 0 06 Feb 2024
Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis Jie Hao Xiaochuan Gong Mingrui Liu 33 7 0 17 Jan 2024
Convex and Non-convex Optimization Under Generalized Smoothness Haochuan Li Jian Qian Yi Tian Alexander Rakhlin Ali Jadbabaie 13 35 0 02 Jun 2023
Convergence of AdaGrad for Non-convex Objectives: Simple Proofs and Relaxed Assumptions Bo Wang Huishuai Zhang Zhirui Ma Wei Chen 40 50 0 29 May 2023
Global Convergence and Stability of Stochastic Gradient Descent V. Patel Shushu Zhang Bowen Tian 33 22 0 04 Oct 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 371 5,818 0 29 Apr 2021
A Simple Convergence Proof of Adam and Adagrad Alexandre Défossez Léon Bottou Francis R. Bach Nicolas Usunier 56 146 0 05 Mar 2020