v1v2v3v4v5v6v7 (latest)

Optimizing Neural Networks with Kronecker-factored Approximate Curvature

19 March 2015

Papers citing "Optimizing Neural Networks with Kronecker-factored Approximate Curvature"

50 / 645 papers shown

Title
Deep Classifiers with Label Noise Modeling and Distance Awareness Vincent Fortuin Mark Collier F. Wenzel J. Allingham J. Liu Dustin Tran Balaji Lakshminarayanan Jesse Berent Rodolphe Jenatton E. Kokiopoulou UQCV 78 11 0 06 Oct 2021
Scale-invariant Learning by Physics Inversion Philipp Holl V. Koltun Nils Thuerey PINN AI4CE 76 9 0 30 Sep 2021
Deep Neural Compression Via Concurrent Pruning and Self-Distillation J. Ó. Neill Sourav Dutta H. Assem VLM 41 5 0 30 Sep 2021
Second-Order Neural ODE Optimizer Guan-Horng Liu T. Chen Evangelos A. Theodorou 77 15 0 29 Sep 2021
Stochastic Training is Not Necessary for Generalization Jonas Geiping Micah Goldblum Phillip E. Pope Michael Moeller Tom Goldstein 175 76 0 29 Sep 2021
Introspective Robot Perception using Smoothed Predictions from Bayesian Neural Networks Jianxiang Feng M. Durner Zoltán-Csaba Márton Ferenc Bálint-Benczédi Rudolph Triebel UQCV BDL 84 11 0 27 Sep 2021
Recent Advances of Continual Learning in Computer Vision: An Overview Haoxuan Qu Hossein Rahmani Li Xu Bryan M. Williams Jun Liu VLM CLL 136 78 0 23 Sep 2021
Training Algorithm Matters for the Performance of Neural Network Potential: A Case Study of Adam and the Kalman Filter Optimizers Yunqi Shao Florian M. Dietrich Carl Nettelblad Chao Zhang 37 12 0 08 Sep 2021
Revisiting Recursive Least Squares for Training Deep Neural Networks Chunyuan Zhang Qi Song Hui Zhou Yi-gui Ou Hongyao Deng Laurence Yang ODL 24 7 0 07 Sep 2021
Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization Alexandre Ramé Corentin Dancette Matthieu Cord OOD 146 211 0 07 Sep 2021
Analytic natural gradient updates for Cholesky factor in Gaussian variational approximation Linda S. L. Tan 108 13 0 01 Sep 2021
Learning Practically Feasible Policies for Online 3D Bin Packing Hang Zhao Chenyang Zhu Xin Xu Hui Huang Kai Xu OffRL 89 84 0 31 Aug 2021
The Number of Steps Needed for Nonconvex Optimization of a Deep Learning Optimizer is a Rational Function of Batch Size Hideaki Iiduka 79 2 0 26 Aug 2021
Batch Normalization Preconditioning for Neural Network Training Susanna Lange Kyle E. Helfrich Qiang Ye 68 9 0 02 Aug 2021
Sparse Bayesian Deep Learning for Dynamic System Identification Hongpeng Zhou Chahine Ibrahim W. Zheng Wei Pan BDL 48 26 0 27 Jul 2021
Structured second-order methods via natural gradient descent Wu Lin Frank Nielsen Mohammad Emtiyaz Khan Mark Schmidt ODL 60 10 0 22 Jul 2021
Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks Shaoshuai Shi Lin Zhang Yue Liu 123 9 0 14 Jul 2021
L2M: Practical posterior Laplace approximation with optimization-driven second moment estimation C. Perone Roberto Silveira Thomas S. Paula ODL UQCV 59 2 0 09 Jul 2021
On the Variance of the Fisher Information for Deep Learning Alexander Soen Ke Sun FedML FAtt 71 18 0 09 Jul 2021
M-FAC: Efficient Matrix-Free Approximations of Second-Order Information Elias Frantar Eldar Kurtic Dan Alistarh 88 59 0 07 Jul 2021
A Survey of Uncertainty in Deep Neural Networks J. Gawlikowski Cedrique Rovile Njieutcheu Tassi Mohsin Ali Jongseo Lee Matthias Humt ... R. Roscher Muhammad Shahzad Wen Yang R. Bamler Xiaoxiang Zhu BDL UQCV OOD 242 1,178 0 07 Jul 2021
KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks J. G. Pauloski Qi Huang Lei Huang Shivaram Venkataraman Kyle Chard Ian Foster Zhao-jie Zhang 86 29 0 04 Jul 2021
Analytic Insights into Structure and Rank of Neural Network Hessian Maps Sidak Pal Singh Gregor Bachmann Thomas Hofmann FAtt 107 37 0 30 Jun 2021
Laplace Redux -- Effortless Bayesian Deep Learning Erik A. Daxberger Agustinus Kristiadi Alexander Immer Runa Eschenhagen Matthias Bauer Philipp Hennig BDL UQCV 256 316 0 28 Jun 2021
Rayleigh-Gauss-Newton optimization with enhanced sampling for variational Monte Carlo R. Webber M. Lindsey 117 6 0 19 Jun 2021
NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weights Kirill Bykov Anna Hedström Shinichi Nakajima Marina M.-C. Höhne FAtt 95 35 0 18 Jun 2021
Natural continual learning: success is a journey, not (just) a destination Ta-Chu Kao Kristopher T. Jensen Gido M. van de Ven A. Bernacchia Guillaume Hennequin CLL 117 49 0 15 Jun 2021
Credit Assignment in Neural Networks through Deep Feedback Control Alexander Meulemans Matilde Tristany Farinha Javier García Ordónez Pau Vilimelis Aceituno João Sacramento Benjamin Grewe 102 37 0 15 Jun 2021
NG+ : A Multi-Step Matrix-Product Natural Gradient Method for Deep Learning Minghan Yang Dong Xu Qiwen Cui Zaiwen Wen Pengxiang Xu 48 4 0 14 Jun 2021
LocoProp: Enhancing BackProp via Local Loss Optimization Ehsan Amid Rohan Anil Manfred K. Warmuth ODL 56 20 0 11 Jun 2021
Quantum Speedup of Natural Gradient for Variational Bayes A. Lopatnikova Minh-Ngoc Tran BDL 119 3 0 10 Jun 2021
Pulling back information geometry Georgios Arvanitidis Miguel González Duque Alison Pouplin Dimitris Kalatzis Søren Hauberg DRL 76 17 0 09 Jun 2021
TENGraD: Time-Efficient Natural Gradient Descent with Exact Fisher-Block Inversion Saeed Soori Bugra Can Baourun Mu Mert Gurbuzbalaban M. Dehnavi 98 10 0 07 Jun 2021
Interactive Label Cleaning with Example-based Explanations Stefano Teso A. Bontempelli Fausto Giunchiglia Andrea Passerini 120 48 0 07 Jun 2021
Tensor Normal Training for Deep Learning Models Yi Ren Shiqian Ma 105 28 0 05 Jun 2021
ViViT: Curvature access through the generalized Gauss-Newton's low-rank structure Felix Dangel Lukas Tatzel Philipp Hennig 91 13 0 04 Jun 2021
A Generalizable Approach to Learning Optimizers Diogo Almeida Clemens Winter Jie Tang Wojciech Zaremba AI4CE 93 29 0 02 Jun 2021
Concurrent Adversarial Learning for Large-Batch Training Yong Liu Xiangning Chen Minhao Cheng Cho-Jui Hsieh Yang You ODL 87 13 0 01 Jun 2021
Training With Data Dependent Dynamic Learning Rates Shreyas Saxena Nidhi Vyas D. DeCoste ODL 24 1 0 27 May 2021
Solving the electronic Schrödinger equation for multiple nuclear geometries with weight-sharing deep neural networks Michael Scherbela Rafael Reisenhofer Leon Gerard P. Marquetand Philipp Grohs 76 51 0 18 May 2021
Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective Florin Gogianu Tudor Berariu Mihaela Rosca Claudia Clopath L. Buşoniu Razvan Pascanu 86 56 0 11 May 2021
Dynamic Game Theoretic Neural Optimizer Guan-Horng Liu T. Chen Evangelos A. Theodorou AI4CE 63 4 0 08 May 2021
Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks Hidenori Tanaka D. Kunin 113 31 0 06 May 2021
Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes James Lucas Juhan Bae Michael Ruogu Zhang Stanislav Fort R. Zemel Roger C. Grosse MoMe 242 28 0 22 Apr 2021
Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning Alexander Immer Matthias Bauer Vincent Fortuin Gunnar Rätsch Mohammad Emtiyaz Khan BDL UQCV 153 109 0 11 Apr 2021
Exact Stochastic Second Order Deep Learning F. Mehouachi C. Kasmi ODL 19 1 0 08 Apr 2021
Robust Trust Region for Weakly Supervised Segmentation D. Marin Yuri Boykov UQCV 58 2 0 05 Apr 2021
Exploiting Invariance in Training Deep Neural Networks Chengxi Ye Xiong Zhou Tristan McKinney Yanfeng Liu Qinggang Zhou Fedor Zhdanov 27 4 0 30 Mar 2021
A Distributed Optimisation Framework Combining Natural Gradient with Hessian-Free for Discriminative Sequence Training Adnan Haider Chao Zhang Florian Kreyssig P. Woodland 112 7 0 12 Mar 2021
Second-order step-size tuning of SGD for non-convex optimization Camille Castera Jérôme Bolte Cédric Févotte Edouard Pauwels ODL 58 10 0 05 Mar 2021