ResearchTrend.AI
Sketchy Empirical Natural Gradient Methods for Deep Learning
v1v2v3 (latest)

Sketchy Empirical Natural Gradient Methods for Deep Learning

Minghan Yang
Dong Xu
Zaiwen Wen
Mengyun Chen
Pengxiang Xu
ArXiv (abs)PDFHTML

Papers citing "Sketchy Empirical Natural Gradient Methods for Deep Learning"

8 / 8 papers shown
Title
MKOR: Momentum-Enabled Kronecker-Factor-Based Optimizer Using Rank-1 Updates
MKOR: Momentum-Enabled Kronecker-Factor-Based Optimizer Using Rank-1 Updates
Mohammad Mozaffari
Sikan Li
Zhao Zhang
M. Dehnavi
74
4
0
02 Jun 2023
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch
Kazuki Osawa
Satoki Ishikawa
Rio Yokota
Shigang Li
Torsten Hoefler
ODL
92
15
0
08 May 2023
Brand New K-FACs: Speeding up K-FAC with Online Decomposition Updates
Brand New K-FACs: Speeding up K-FAC with Online Decomposition Updates
C. Puiu
32
2
0
16 Oct 2022
Riemannian Natural Gradient Methods
Riemannian Natural Gradient Methods
Jiang Hu
Ruicheng Ao
Anthony Man-Cho So
Minghan Yang
Zaiwen Wen
69
11
0
15 Jul 2022
Randomized K-FACs: Speeding up K-FAC with Randomized Numerical Linear Algebra
Randomized K-FACs: Speeding up K-FAC with Randomized Numerical Linear Algebra
C. Puiu
68
2
0
30 Jun 2022
Rethinking Exponential Averaging of the Fisher
Rethinking Exponential Averaging of the Fisher
C. Puiu
54
1
0
10 Apr 2022
NG+ : A Multi-Step Matrix-Product Natural Gradient Method for Deep Learning
NG+ : A Multi-Step Matrix-Product Natural Gradient Method for Deep Learning
Minghan Yang
Dong Xu
Qiwen Cui
Zaiwen Wen
Pengxiang Xu
48
4
0
14 Jun 2021
Eigenvalue-corrected Natural Gradient Based on a New Approximation
Eigenvalue-corrected Natural Gradient Based on a New Approximation
Kai-Xin Gao
Xiaolei Liu
Zheng-Hai Huang
Min Wang
Shuangling Wang
Zidong Wang
Dachuan Xu
F. Yu
ODL
51
7
0
27 Nov 2020