Grams: Gradient Descent with Adaptive Momentum Scaling

v1v2v3 (latest)

Grams: Gradient Descent with Adaptive Momentum Scaling

22 December 2024

ArXiv (abs)PDF HTML

Papers citing "Grams: Gradient Descent with Adaptive Momentum Scaling"

6 / 6 papers shown

Title
Improving Adaptive Moment Optimization via Preconditioner Diagonalization Son Nguyen B. Liu Lizhang Chen Qiang Liu ODL 161 3 0 11 Feb 2025
Video Latent Flow Matching: Optimal Polynomial Projections for Video Interpolation and Extrapolation Yang Cao Zhao Song Chiwun Yang VGen 149 3 0 01 Feb 2025
Cautious Optimizers: Improving Training with One Line of Code Kaizhao Liang Lizhang Chen B. Liu Qiang Liu ODL 261 9 0 25 Nov 2024
Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent Bo Chen Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao Song 152 22 0 15 Oct 2024
SOAP: Improving and Stabilizing Shampoo using Adam Nikhil Vyas Depen Morwani Rosie Zhao Itai Shapira David Brandfonbrener Lucas Janson Sham Kakade Sham Kakade 165 38 0 17 Sep 2024
Binary Hypothesis Testing for Softmax Models and Leverage Score Models Yeqi Gao Yuzhou Gu Zhao Song 75 0 0 09 May 2024