SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients

SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients

15 June 2021

Heng-Chiao Huang

Papers citing "SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients"

11 / 11 papers shown

Title
HOME-3: High-Order Momentum Estimator with Third-Power Gradient for Convex and Smooth Nonconvex Optimization Wei Zhang Arif Hassan Zidan Afrar Jahin Wei Zhang Tianming Liu ODL 42 0 0 16 May 2025
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions Yusu Hong Junhong Lin 76 13 0 06 Feb 2024
Enhanced Adaptive Gradient Algorithms for Nonconvex-PL Minimax Optimization Feihu Huang Chunyu Xuan Xinrui Wang Siqi Zhang Songcan Chen 52 7 0 07 Mar 2023
Adam $^+$ : A Stochastic Method with Adaptive Variance Reduction Mingrui Liu Wei Zhang Francesco Orabona Tianbao Yang 34 28 0 24 Nov 2020
AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients Juntang Zhuang Tommy M. Tang Yifan Ding S. Tatikonda Nicha Dvornek X. Papademetris James S. Duncan ODL 83 507 0 15 Oct 2020
Hybrid Stochastic Gradient Descent Algorithms for Stochastic Nonconvex Optimization Quoc Tran-Dinh Nhan H. Pham Dzung Phan Lam M. Nguyen 47 55 0 15 May 2019
Adaptive Gradient Methods with Dynamic Bound of Learning Rate Liangchen Luo Yuanhao Xiong Yan Liu Xu Sun ODL 46 600 0 26 Feb 2019
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator Cong Fang C. J. Li Zhouchen Lin Tong Zhang 81 572 0 04 Jul 2018
Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks Jinghui Chen Dongruo Zhou Yiqi Tang Ziyan Yang Yuan Cao Quanquan Gu ODL 63 192 0 18 Jun 2018
Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming Saeed Ghadimi Guanghui Lan ODL 71 1,538 0 22 Sep 2013
ADADELTA: An Adaptive Learning Rate Method Matthew D. Zeiler ODL 108 6,619 0 22 Dec 2012