v1v2v3v4v5v6v7 (latest)

Optimizing Neural Networks with Kronecker-factored Approximate Curvature

19 March 2015

Papers citing "Optimizing Neural Networks with Kronecker-factored Approximate Curvature"

50 / 645 papers shown

Title
Generalized Variational Inference in Function Spaces: Gaussian Measures meet Bayesian Deep Learning Veit Wild Robert Hu Dino Sejdinovic BDL 135 13 0 12 May 2022
A Novel Fast Exact Subproblem Solver for Stochastic Quasi-Newton Cubic Regularized Optimization Jarad Forristal J. Griffin Wenwen Zhou S. Yektamaram ODL 26 0 0 19 Apr 2022
Deep Unlearning via Randomized Conditionally Independent Hessians Ronak R. Mehta Sourav Pal Vikas Singh Sathya Ravi MU 76 89 0 15 Apr 2022
Rethinking Exponential Averaging of the Fisher C. Puiu 54 1 0 10 Apr 2022
Weight Matrix Dimensionality Reduction in Deep Learning via Kronecker Multi-layer Architectures Jarom D. Hogue Robert M. Kirby A. Narayan 61 0 0 08 Apr 2022
Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Network Byung-Kwan Lee Junho Kim Y. Ro AAML 59 20 0 06 Apr 2022
Learning to Accelerate by the Methods of Step-size Planning Hengshuai Yao 98 0 0 01 Apr 2022
Practical tradeoffs between memory, compute, and performance in learned optimizers Luke Metz C. Freeman James Harrison Niru Maheswaranathan Jascha Narain Sohl-Dickstein 148 32 0 22 Mar 2022
Half-Inverse Gradients for Physical Deep Learning Patrick Schnell Philipp Holl Nils Thuerey 57 8 0 18 Mar 2022
Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers Guodong Zhang Aleksandar Botev James Martens OffRL 83 28 0 15 Mar 2022
Model-agnostic out-of-distribution detection using combined statistical tests Federico Bergamin Pierre-Alexandre Mattei Jakob Drachmann Havtorn Hugo Senetaire Hugo Schmutz Lars Maaløe Søren Hauberg J. Frellsen OODD 87 19 0 02 Mar 2022
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 118 14 0 28 Feb 2022
Bayesian Model Selection, the Marginal Likelihood, and Generalization Sanae Lotfi Pavel Izmailov Gregory W. Benton Micah Goldblum A. Wilson UQCV BDL 155 58 0 23 Feb 2022
Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations Alexander Immer Tycho F. A. van der Ouderaa Gunnar Rätsch Vincent Fortuin Mark van der Wilk BDL 147 48 0 22 Feb 2022
Myriad: a real-world testbed to bridge trajectory optimization and deep learning Nikolaus H. R. Howe Simon Dufort-Labbé Nitarshan Rajkumar Pierre-Luc Bacon 72 5 0 22 Feb 2022
Efficient Natural Gradient Descent Methods for Large-Scale PDE-Based Optimization Problems L. Nurbekyan Wanzhou Lei Yunbo Yang 68 12 0 13 Feb 2022
A Geometric Understanding of Natural Gradient Qinxun Bai S. Rosenberg Wei Xu 70 2 0 13 Feb 2022
A Mini-Block Fisher Method for Deep Neural Networks Achraf Bahamou Shiqian Ma Yi Ren ODL 94 9 0 08 Feb 2022
Approximating Full Conformal Prediction at Scale via Influence Functions Javier Abad Umang Bhatt Adrian Weller Giovanni Cherubin 126 10 0 02 Feb 2022
A Stochastic Bundle Method for Interpolating Networks Alasdair Paren Leonard Berrada Rudra P. K. Poudel M. P. Kumar 76 4 0 29 Jan 2022
Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization Frederik Benzing ODL 127 26 0 28 Jan 2022
Visualizing the Diversity of Representations Learned by Bayesian Neural Networks Dennis Grinwald Kirill Bykov Shinichi Nakajima Marina M.-C. Höhne 93 5 0 26 Jan 2022
Efficient Approximations of the Fisher Matrix in Neural Networks using Kronecker Product Singular Value Decomposition Abdoulaye Koroko A. Anciaux-Sedrakian I. B. Gharbia Valérie Garès M. Haddou Quang-Huy Tran 97 7 0 25 Jan 2022
Weight Expansion: A New Perspective on Dropout and Generalization Gao Jin Xinping Yi Pengfei Yang Lijun Zhang S. Schewe Xiaowei Huang 92 5 0 23 Jan 2022
Understanding the Effects of Second-Order Approximations in Natural Policy Gradient Reinforcement Learning Brennan Gebotys Alexander Wong David A Clausi 39 2 0 22 Jan 2022
UWC: Unit-wise Calibration Towards Rapid Network Compression Chen Lin Zheyang Li Bo Peng Haoji Hu Wenming Tan Ye Ren Shiliang Pu MQ 41 1 0 17 Jan 2022
Recursive Least Squares Advantage Actor-Critic Algorithms Yuan Wang Chunyuan Zhang Tianzong Yu Meng-tao Ma 43 0 0 15 Jan 2022
$$m^\ast$ of two-dimensional electron gas: a neural canonical transformation study$ $m^\ast$ of two-dimensional electron gas: a neural canonical transformation study H.-j. Xie Linfeng Zhang Lei Wang 91 8 0 10 Jan 2022
Dynamically Stable Poincaré Embeddings for Neural Manifolds Jun Chen Yuang Liu Xiangrui Zhao Mengmeng Wang Yang Liu 63 0 0 21 Dec 2021
GOSH: Task Scheduling Using Deep Surrogate Models in Fog Computing Environments Shreshth Tuli G. Casale N. Jennings 73 21 0 16 Dec 2021
Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time Zhao Song Licheng Zhang Ruizhe Zhang 116 66 0 14 Dec 2021
SCORE: Approximating Curvature Information under Self-Concordant Regularization Adeyemi Damilare Adeoye Alberto Bemporad 44 4 0 14 Dec 2021
Minimization of Stochastic First-order Oracle Complexity of Adaptive Methods for Nonconvex Optimization Hideaki Iiduka 43 0 0 14 Dec 2021
Effective dimension of machine learning models Amira Abbas David Sutter Alessio Figalli Stefan Woerner 121 18 0 09 Dec 2021
Explicitly antisymmetrized neural network layers for variational Monte Carlo simulation Jeffmin Lin Gil Goldshlager Lin Lin 79 25 0 07 Dec 2021
Regularized Newton Method with Global $O(1/k^2)$ Convergence Konstantin Mishchenko 89 41 0 03 Dec 2021
AutoDrop: Training Deep Learning Models with Automatic Learning Rate Drop Yunfei Teng Jing Wang A. Choromańska 82 2 0 30 Nov 2021
Depth Without the Magic: Inductive Bias of Natural Gradient Descent A. Kerekes Anna Mészáros Ferenc Huszár ODL 48 4 0 22 Nov 2021
Merging Models with Fisher-Weighted Averaging Michael Matena Colin Raffel FedML MoMe 116 403 0 18 Nov 2021
PredProp: Bidirectional Stochastic Optimization with Precision Weighted Predictive Coding André Ofner Sebastian Stober 47 2 0 16 Nov 2021
Neuron-based Pruning of Deep Neural Networks with Better Generalization using Kronecker Factored Curvature Approximation Abdolghani Ebrahimi Diego Klabjan 34 4 0 16 Nov 2021
Kronecker Factorization for Preventing Catastrophic Forgetting in Large-scale Medical Entity Linking Denis Jered McInerney Luyang Kong Kristjan Arumae Byron C. Wallace Parminder Bhatia CLL 43 1 0 11 Nov 2021
Linear Speedup in Personalized Collaborative Learning El Mahdi Chayti Sai Praneeth Karimireddy Sebastian U. Stich Nicolas Flammarion Martin Jaggi FedML 74 13 0 10 Nov 2021
Estimating High Order Gradients of the Data Distribution by Denoising Chenlin Meng Yang Song Wenzhe Li Stefano Ermon DiffM 94 46 0 08 Nov 2021
Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learning Runa Eschenhagen Erik A. Daxberger Philipp Hennig Agustinus Kristiadi UQCV BDL 73 23 0 05 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 90 15 0 01 Nov 2021
Does the Data Induce Capacity Control in Deep Learning? Rubing Yang Jialin Mao Pratik Chaudhari 126 16 0 27 Oct 2021
Gradient representations in ReLU networks as similarity functions Dániel Rácz Balint Daroczy FAtt 57 1 0 26 Oct 2021
Sharpness-Aware Minimization Improves Language Model Generalization Dara Bahri H. Mobahi Yi Tay 182 104 0 16 Oct 2021
Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions Nicholas Gao Stephan Günnemann 86 40 0 11 Oct 2021