ResearchTrend.AI
Optimal Diagonal Preconditioning

Optimal Diagonal Preconditioning

Zhaonan Qu
Wenzhi Gao
Oliver Hinder
Yinyu Ye
Zhengyuan Zhou
ArXivPDFHTML

Papers citing "Optimal Diagonal Preconditioning"

5 / 5 papers shown
Title
Towards Quantifying the Hessian Structure of Neural Networks
Towards Quantifying the Hessian Structure of Neural Networks
Zhaorui Dong
Yushun Zhang
Zhi-Quan Luo
Jianfeng Yao
Ruoyu Sun
33
0
0
05 May 2025
Weight Conditioning for Smooth Optimization of Neural Networks
Weight Conditioning for Smooth Optimization of Neural Networks
Hemanth Saratchandran
Thomas X. Wang
Simon Lucey
51
1
0
05 Sep 2024
Adam-mini: Use Fewer Learning Rates To Gain More
Adam-mini: Use Fewer Learning Rates To Gain More
Yushun Zhang
Congliang Chen
Ziniu Li
Tian Ding
Chenwei Wu
Yinyu Ye
Zhi-Quan Luo
Ruoyu Sun
46
38
0
24 Jun 2024
Large Batch Analysis for Adagrad Under Anisotropic Smoothness
Large Batch Analysis for Adagrad Under Anisotropic Smoothness
Yuxing Liu
Rui Pan
Tong Zhang
30
6
0
21 Jun 2024
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking
Frederik Kunstner
V. S. Portella
Mark Schmidt
Nick Harvey
33
8
0
05 Jun 2023