ResearchTrend.AI
Optimizing Neural Networks with Kronecker-factored Approximate Curvature
v1v2v3v4v5v6v7 (latest)

Optimizing Neural Networks with Kronecker-factored Approximate Curvature

James Martens
Roger C. Grosse
    ODL
ArXiv (abs)PDFHTML

Papers citing "Optimizing Neural Networks with Kronecker-factored Approximate Curvature"

50 / 645 papers shown
Title
Incremental Gauss-Newton Descent for Machine Learning
Incremental Gauss-Newton Descent for Machine Learning
Mikalai Korbit
Mario Zanon
ODL
45
0
0
10 Aug 2024
Towards the Spectral bias Alleviation by Normalizations in Coordinate Networks
Towards the Spectral bias Alleviation by Normalizations in Coordinate Networks
Zhicheng Cai
Hao Zhu
Qiu Shen
Xinran Wang
Xun Cao
114
0
0
25 Jul 2024
Correlations Are Ruining Your Gradient Descent
Correlations Are Ruining Your Gradient Descent
Nasir Ahmad
121
3
0
15 Jul 2024
Stepping on the Edge: Curvature Aware Learning Rate Tuners
Stepping on the Edge: Curvature Aware Learning Rate Tuners
Vincent Roulet
Atish Agarwala
Jean-Bastien Grill
Grzegorz Swirszcz
Mathieu Blondel
Fabian Pedregosa
99
3
0
08 Jul 2024
A New Perspective on Shampoo's Preconditioner
A New Perspective on Shampoo's Preconditioner
Depen Morwani
Itai Shapira
Nikhil Vyas
Eran Malach
Sham Kakade
Lucas Janson
113
12
0
25 Jun 2024
Adam-mini: Use Fewer Learning Rates To Gain More
Adam-mini: Use Fewer Learning Rates To Gain More
Yushun Zhang
Congliang Chen
Ziniu Li
Tian Ding
Chenwei Wu
Yinyu Ye
Zhi-Quan Luo
Ruoyu Sun
136
58
0
24 Jun 2024
EVCL: Elastic Variational Continual Learning with Weight Consolidation
EVCL: Elastic Variational Continual Learning with Weight Consolidation
Hunar Batra
Ronald Clark
CLL
104
2
0
23 Jun 2024
An Improved Empirical Fisher Approximation for Natural Gradient Descent
An Improved Empirical Fisher Approximation for Natural Gradient Descent
Xiaodong Wu
Wenyi Yu
Chao Zhang
Philip Woodland
84
5
0
10 Jun 2024
Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning
Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning
Mohamed Elsayed
Homayoon Farrahi
Felix Dangel
A. Rupam Mahmood
73
4
0
05 Jun 2024
Improving Generalization and Convergence by Enhancing Implicit Regularization
Improving Generalization and Convergence by Enhancing Implicit Regularization
Mingze Wang
Haotian He
Jinbo Wang
Zilin Wang
Guanhua Huang
Feiyu Xiong
Zhiyu Li
E. Weinan
Lei Wu
100
8
0
31 May 2024
Scalable Bayesian Learning with posteriors
Scalable Bayesian Learning with posteriors
Samuel Duffield
Kaelan Donatella
Johnathan Chiu
Phoebe Klett
Daniel Simpson
BDLUQCV
182
2
0
31 May 2024
Ground state phases of the two-dimension electron gas with a unified variational approach
Ground state phases of the two-dimension electron gas with a unified variational approach
Conor Smith
Yixiao Chen
Ryan Levy
Yubo Yang
Miguel A. Morales
Shiwei Zhang
OT
43
4
0
29 May 2024
4-bit Shampoo for Memory-Efficient Network Training
4-bit Shampoo for Memory-Efficient Network Training
Sike Wang
Jia Li
Pan Zhou
Hua Huang
MQ
155
9
0
28 May 2024
From Learning to Optimize to Learning Optimization Algorithms
From Learning to Optimize to Learning Optimization Algorithms
Camille Castera
Peter Ochs
189
1
0
28 May 2024
Calibrated Dataset Condensation for Faster Hyperparameter Search
Calibrated Dataset Condensation for Faster Hyperparameter Search
Mucong Ding
Yuancheng Xu
Tahseen Rabbani
Xiaoyu Liu
Brian J Gravelle
Teresa M. Ranadive
Tai-Ching Tuan
Furong Huang
DD
77
0
0
27 May 2024
AdaFisher: Adaptive Second Order Optimization via Fisher Information
AdaFisher: Adaptive Second Order Optimization via Fisher Information
Damien Martins Gomes
Yanlei Zhang
Eugene Belilovsky
Guy Wolf
Mahdi S. Hosseini
ODL
218
3
0
26 May 2024
Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks
Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks
Felix Dangel
Johannes Müller
Marius Zeinhofer
ODL
107
11
0
24 May 2024
Fisher Flow Matching for Generative Modeling over Discrete Data
Fisher Flow Matching for Generative Modeling over Discrete Data
Oscar Davis
Samuel Kessler
Mircea Petrache
.Ismail .Ilkan Ceylan
Michael M. Bronstein
A. Bose
111
21
0
23 May 2024
What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions
What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions
Sang Keun Choe
Hwijeen Ahn
Juhan Bae
Kewen Zhao
Minsoo Kang
...
Teruko Mitamura
Jeff Schneider
Eduard Hovy
Roger C. Grosse
Eric Xing
TDI
97
44
0
22 May 2024
Thermodynamic Natural Gradient Descent
Thermodynamic Natural Gradient Descent
Kaelan Donatella
Samuel Duffield
Maxwell Aifer
Denis Melanson
Gavin Crooks
Patrick J. Coles
55
3
0
22 May 2024
G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translation
G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translation
Xingyuan Pan
Luyang Huang
Liyan Kang
Zhicheng Liu
Yu Lu
Shanbo Cheng
ALM
135
15
0
21 May 2024
Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows
Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows
Sibylle Marcotte
Rémi Gribonval
Gabriel Peyré
58
1
0
21 May 2024
FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information
FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information
Dongseong Hwang
ODL
97
9
0
21 May 2024
Training Data Attribution via Approximate Unrolled Differentiation
Training Data Attribution via Approximate Unrolled Differentiation
Juhan Bae
Wu Lin
Jonathan Lorraine
Roger C. Grosse
TDIMU
156
18
0
20 May 2024
Using Degeneracy in the Loss Landscape for Mechanistic Interpretability
Using Degeneracy in the Loss Landscape for Mechanistic Interpretability
Lucius Bushnaq
Jake Mendel
Stefan Heimersheim
Dan Braun
Nicholas Goldowsky-Dill
Kaarel Hänni
Cindy Wu
Marius Hobbhahn
88
7
0
17 May 2024
Transferable Neural Wavefunctions for Solids
Transferable Neural Wavefunctions for Solids
Leon Gerard
Michael Scherbela
H. Sutterud
Matthew Foulkes
Philipp Grohs
64
4
0
13 May 2024
Q-Newton: Hybrid Quantum-Classical Scheduling for Accelerating Neural Network Training with Newton's Gradient Descent
Q-Newton: Hybrid Quantum-Classical Scheduling for Accelerating Neural Network Training with Newton's Gradient Descent
Pingzhi Li
Junyu Liu
Hanrui Wang
Tianlong Chen
204
2
0
30 Apr 2024
Regularized Gauss-Newton for Optimizing Overparameterized Neural Networks
Regularized Gauss-Newton for Optimizing Overparameterized Neural Networks
Adeyemi Damilare Adeoye
Philipp Christian Petersen
Alberto Bemporad
65
1
0
23 Apr 2024
QGen: On the Ability to Generalize in Quantization Aware Training
QGen: On the Ability to Generalize in Quantization Aware Training
Mohammadhossein Askarihemmat
Ahmadreza Jeddi
Reyhane Askari Hemmat
Ivan Lazarevich
Alexander Hoffman
Sudhakar Sah
Ehsan Saboori
Yvon Savaria
Jean-Pierre David
MQ
99
1
0
17 Apr 2024
Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classification
Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classification
Denis Huseljic
Paul Hahn
M. Herde
Lukas Rauch
Bernhard Sick
111
2
0
13 Apr 2024
Variational Stochastic Gradient Descent for Deep Neural Networks
Variational Stochastic Gradient Descent for Deep Neural Networks
Haotian Chen
Anna Kuzina
Babak Esmaeili
Jakub M. Tomczak
109
0
0
09 Apr 2024
Stochastic Online Optimization for Cyber-Physical and Robotic Systems
Stochastic Online Optimization for Cyber-Physical and Robotic Systems
Hao Ma
Melanie Zeilinger
Michael Muehlebach
92
1
0
08 Apr 2024
A Structure-Guided Gauss-Newton Method for Shallow ReLU Neural Network
A Structure-Guided Gauss-Newton Method for Shallow ReLU Neural Network
Zhiqiang Cai
Tong Ding
Min Liu
Xinyu Liu
Jianlin Xia
414
2
0
07 Apr 2024
FedFisher: Leveraging Fisher Information for One-Shot Federated Learning
FedFisher: Leveraging Fisher Information for One-Shot Federated Learning
Divyansh Jhunjhunwala
Shiqiang Wang
Gauri Joshi
FedML
73
13
0
19 Mar 2024
PETScML: Second-order solvers for training regression problems in Scientific Machine Learning
PETScML: Second-order solvers for training regression problems in Scientific Machine Learning
Stefano Zampini
Umberto Zerbinati
George Turkyyiah
David E. Keyes
67
5
0
18 Mar 2024
FAGH: Accelerating Federated Learning with Approximated Global Hessian
FAGH: Accelerating Federated Learning with Approximated Global Hessian
Mrinmay Sen
A. K. Qin
Krishna Mohan
FedML
85
1
0
16 Mar 2024
Function-space Parameterization of Neural Networks for Sequential Learning
Function-space Parameterization of Neural Networks for Sequential Learning
Aidan Scannell
Riccardo Mereu
Paul E. Chang
Ella Tamir
Joni Pajarinen
Arno Solin
BDL
92
5
0
16 Mar 2024
Variation Due to Regularization Tractably Recovers Bayesian Deep Learning
Variation Due to Regularization Tractably Recovers Bayesian Deep Learning
James McInerney
Nathan Kallus
BDLUQCV
30
0
0
15 Mar 2024
Towards Faster Training of Diffusion Models: An Inspiration of A Consistency Phenomenon
Towards Faster Training of Diffusion Models: An Inspiration of A Consistency Phenomenon
Tianshuo Xu
Peng Mi
Ruilin Wang
Yingcong Chen
DiffM
162
6
0
14 Mar 2024
A Framework for Strategic Discovery of Credible Neural Network Surrogate Models under Uncertainty
A Framework for Strategic Discovery of Credible Neural Network Surrogate Models under Uncertainty
Pratyush Kumar Singh
Kathryn A. Farrell-Maupin
D. Faghihi
78
8
0
13 Mar 2024
On Representing Electronic Wave Functions with Sign Equivariant Neural Networks
On Representing Electronic Wave Functions with Sign Equivariant Neural Networks
Nicholas Gao
Stephan Günnemann
62
3
0
08 Mar 2024
Continual Learning and Catastrophic Forgetting
Continual Learning and Catastrophic Forgetting
Gido M. van de Ven
Nicholas Soures
Dhireesha Kudithipudi
94
58
0
08 Mar 2024
Inverse-Free Fast Natural Gradient Descent Method for Deep Learning
Inverse-Free Fast Natural Gradient Descent Method for Deep Learning
Xinwei Ou
Ce Zhu
Xiaolin Huang
Yipeng Liu
ODL
136
0
0
06 Mar 2024
SOFIM: Stochastic Optimization Using Regularized Fisher Information Matrix
SOFIM: Stochastic Optimization Using Regularized Fisher Information Matrix
Mrinmay Sen
A. K. Qin
Gayathri C
Raghu Kishore N
Yen-Wei Chen
Balasubramanian Raman
64
2
0
05 Mar 2024
Why Transformers Need Adam: A Hessian Perspective
Why Transformers Need Adam: A Hessian Perspective
Yushun Zhang
Congliang Chen
Tian Ding
Ziniu Li
Ruoyu Sun
Zhimin Luo
132
57
0
26 Feb 2024
Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihood
Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihood
Rayen Dhahri
Alexander Immer
Bertrand Charpentier
Stephan Günnemann
Vincent Fortuin
BDLUQCV
85
5
0
25 Feb 2024
Stochastic Hessian Fittings with Lie Groups
Stochastic Hessian Fittings with Lie Groups
Xi-Lin Li
144
1
0
19 Feb 2024
Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning
Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning
Jason Yoo
Yunpeng Liu
Frank Wood
Geoff Pleiss
CLL
62
5
0
14 Feb 2024
Active Few-Shot Fine-Tuning
Active Few-Shot Fine-Tuning
Jonas Hübotter
Bhavya Sukhija
Lenart Treven
Yarden As
Andreas Krause
104
1
0
13 Feb 2024
Preconditioners for the Stochastic Training of Neural Fields
Preconditioners for the Stochastic Training of Neural Fields
Shin-Fang Chng
Hemanth Saratchandran
Simon Lucey
99
1
0
13 Feb 2024

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.