Optimal Diagonal Preconditioning

Optimal Diagonal Preconditioning

2 September 2022

Wenzhi Gao

Papers citing "Optimal Diagonal Preconditioning"

5 / 5 papers shown

Title
Towards Quantifying the Hessian Structure of Neural Networks Zhaorui Dong Yushun Zhang Zhi-Quan Luo Jianfeng Yao Ruoyu Sun 33 0 0 05 May 2025
Weight Conditioning for Smooth Optimization of Neural Networks Hemanth Saratchandran Thomas X. Wang Simon Lucey 51 1 0 05 Sep 2024
Adam-mini: Use Fewer Learning Rates To Gain More Yushun Zhang Congliang Chen Ziniu Li Tian Ding Chenwei Wu Yinyu Ye Zhi-Quan Luo Ruoyu Sun 46 38 0 24 Jun 2024
Large Batch Analysis for Adagrad Under Anisotropic Smoothness Yuxing Liu Rui Pan Tong Zhang 28 6 0 21 Jun 2024
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking Frederik Kunstner V. S. Portella Mark Schmidt Nick Harvey 33 8 0 05 Jun 2023