Gradient Methods with Online Scaling Part I. Theoretical Foundations

29 May 2025

Papers citing "Gradient Methods with Online Scaling Part I. Theoretical Foundations"

22 / 22 papers shown

Title
Gradient Methods with Online Scaling Wenzhi Gao Ya-Chi Chu Yinyu Ye Madeleine Udell 78 4 0 04 Nov 2024
Online Learning Guided Quasi-Newton Methods with Global Non-Asymptotic Convergence Ruichen Jiang Aryan Mokhtari 13 4 0 03 Oct 2024
Adam-mini: Use Fewer Learning Rates To Gain More Yushun Zhang Congliang Chen Ziniu Li Tian Ding Chenwei Wu Yinyu Ye Zhi-Quan Luo Ruoyu Sun 103 56 0 24 Jun 2024
MADA: Meta-Adaptive Optimizers through hyper-gradient Descent Kaan Ozkara Can Karakus Parameswaran Raman Mingyi Hong Shoham Sabach Branislav Kveton Volkan Cevher 77 3 0 17 Jan 2024
A simple uniformly optimal method without line search for convex optimization Tianjiao Li Guanghui Lan 72 22 0 16 Oct 2023
Adaptive Proximal Gradient Method for Convex Optimization Yura Malitsky Konstantin Mishchenko 50 24 0 04 Aug 2023
Unconstrained Online Learning with Unbounded Losses Andrew Jacobsen Ashok Cutkosky 55 18 0 08 Jun 2023
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking Frederik Kunstner V. S. Portella Mark Schmidt Nick Harvey 73 10 0 05 Jun 2023
Online Learning Guided Curvature Approximation: A Quasi-Newton Method with Global Non-Asymptotic Superlinear Convergence Ruichen Jiang Qiujiang Jin Aryan Mokhtari 41 19 0 16 Feb 2023
No-Regret Dynamics in the Fenchel Game: A Unified Framework for Algorithmic Convex Optimization Jun-Kun Wang Jacob D. Abernethy Kfir Y. Levy 70 23 0 22 Nov 2021
AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients Juntang Zhuang Tommy M. Tang Yifan Ding S. Tatikonda Nicha Dvornek X. Papademetris James S. Duncan ODL 165 517 0 15 Oct 2020
Non-asymptotic Superlinear Convergence of Standard Quasi-Newton Methods Qiujiang Jin Aryan Mokhtari 49 42 0 30 Mar 2020
Disentangling Adaptive Gradient Methods from Learning Rates Naman Agarwal Rohan Anil Elad Hazan Tomer Koren Cyril Zhang 90 38 0 26 Feb 2020
A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance Xiaoyun Li Zhenxun Zhuang Francesco Orabona 58 19 0 12 Feb 2020
Introduction to Online Convex Optimization Elad Hazan OffRL 175 1,932 0 07 Sep 2019
On the Convergence of Adam and Beyond Sashank J. Reddi Satyen Kale Surinder Kumar 99 2,504 0 19 Apr 2019
Surrogate Losses for Online Learning of Stepsizes in Stochastic Non-Convex Optimization Zhenxun Zhuang Ashok Cutkosky Francesco Orabona 77 5 0 25 Jan 2019
Shampoo: Preconditioned Stochastic Tensor Optimization Vineet Gupta Tomer Koren Y. Singer ODL 90 224 0 26 Feb 2018
Online Learning Rate Adaptation with Hypergradient Descent A. G. Baydin R. Cornish David Martínez-Rubio Mark Schmidt Frank Wood ODL 77 247 0 14 Mar 2017
Coin Betting and Parameter-Free Online Learning Francesco Orabona D. Pál 162 166 0 12 Feb 2016
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.9K 150,260 0 22 Dec 2014
Adaptive Bound Optimization for Online Convex Optimization H. B. McMahan Matthew J. Streeter ODL 101 390 0 26 Feb 2010