Improved Analysis of Clipping Algorithms for Non-convex Optimization

Improved Analysis of Clipping Algorithms for Non-convex Optimization

5 October 2020

Cong Fang

Papers citing "Improved Analysis of Clipping Algorithms for Non-convex Optimization"

12 / 12 papers shown

Title
AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training Huishuai Zhang Bohan Wang Luoxin Chen ODL 47 0 0 22 May 2025
Extended convexity and smoothness and their applications in deep learning Binchuan Qi Wei Gong Li Li 68 0 0 08 Oct 2024
An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness Xiaochuan Gong Jie Hao Mingrui Liu 82 2 0 28 Sep 2024
Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance Qi Zhang Yi Zhou Shaofeng Zou 71 5 0 01 Apr 2024
Directional Smoothness and Gradient Methods: Convergence and Adaptivity Aaron Mishkin Ahmed Khaled Yuanhao Wang Aaron Defazio Robert Mansel Gower 62 7 0 06 Mar 2024
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions Yusu Hong Junhong Lin 63 13 0 06 Feb 2024
Regularized Q-Learning with Linear Function Approximation Jiachen Xi Alfredo Garcia P. Momcilovic 62 2 0 26 Jan 2024
Why are Adaptive Methods Good for Attention Models? J.N. Zhang Sai Praneeth Karimireddy Andreas Veit Seungyeon Kim Sashank J. Reddi Surinder Kumar S. Sra 72 80 0 06 Dec 2019
Quasi-hyperbolic momentum and Adam for deep learning Jerry Ma Denis Yarats ODL 97 129 0 16 Oct 2018
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator Cong Fang C. J. Li Zhouchen Lin Tong Zhang 73 572 0 04 Jul 2018
Snapshot Ensembles: Train 1, get M for free Gao Huang Yixuan Li Geoff Pleiss Zhuang Liu John E. Hopcroft Kilian Q. Weinberger OOD FedML UQCV 88 938 0 01 Apr 2017
Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming Saeed Ghadimi Guanghui Lan ODL 32 1,538 0 22 Sep 2013

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.