Towards Quantifying the Preconditioning Effect of Adam

Towards Quantifying the Preconditioning Effect of Adam

11 February 2024

Sujay Sanghavi

Inderjit S. Dhillon

Papers citing "Towards Quantifying the Preconditioning Effect of Adam"

6 / 6 papers shown

Title
Towards Quantifying the Hessian Structure of Neural Networks Zhaorui Dong Yushun Zhang Zhi-Quan Luo Jianfeng Yao Ruoyu Sun 31 0 0 05 May 2025
Gradient Methods with Online Scaling Wenzhi Gao Ya-Chi Chu Yinyu Ye Madeleine Udell 38 1 0 04 Nov 2024
Adam-mini: Use Fewer Learning Rates To Gain More Yushun Zhang Congliang Chen Ziniu Li Tian Ding Chenwei Wu Yinyu Ye Zhi-Quan Luo Ruoyu Sun 46 37 0 24 Jun 2024
Large Batch Analysis for Adagrad Under Anisotropic Smoothness Yuxing Liu Rui Pan Tong Zhang 26 5 0 21 Jun 2024
A Simple Convergence Proof of Adam and Adagrad Alexandre Défossez Léon Bottou Francis R. Bach Nicolas Usunier 56 144 0 05 Mar 2020
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark W. Schmidt 139 1,201 0 16 Aug 2016