Eigenvalue-corrected Natural Gradient Based on a New Approximation

27 November 2020

Papers citing "Eigenvalue-corrected Natural Gradient Based on a New Approximation"

3 / 3 papers shown

Title
Gradient Multi-Normalization for Stateless and Scalable LLM Training M. Scetbon Chao Ma Wenbo Gong Edward Meeds 183 1 0 10 Feb 2025
An Improved Empirical Fisher Approximation for Natural Gradient Descent Xiaodong Wu Wenyi Yu Chao Zhang Philip Woodland 84 5 0 10 Jun 2024
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 229 169 0 03 Jul 2020