Analytic natural gradient updates for Cholesky factor in Gaussian variational approximation

1 September 2021

Papers citing "Analytic natural gradient updates for Cholesky factor in Gaussian variational approximation"

41 / 41 papers shown

Title
Spectral-factorized Positive-definite Curvature Learning for NN Training Wu Lin Felix Dangel Runa Eschenhagen Juhan Bae Richard E. Turner Roger B. Grosse 138 0 0 10 Feb 2025
normflows: A PyTorch Package for Normalizing Flows Vincent Stimper David Liu Andrew Campbell V. Berenz Lukas Ryll Bernhard Schölkopf José Miguel Hernández-Lobato AI4CE 60 63 0 26 Jan 2023
Tractable structured natural gradient descent using local parameterizations Wu Lin Frank Nielsen Mohammad Emtiyaz Khan Mark Schmidt 59 30 0 15 Feb 2021
A Simple Convergence Proof of Adam and Adagrad Alexandre Défossez Léon Bottou Francis R. Bach Nicolas Usunier 112 155 0 05 Mar 2020
Handling the Positive-Definite Constraint in the Bayesian Learning Rule Wu Lin Mark Schmidt Mohammad Emtiyaz Khan BDL 73 36 0 24 Feb 2020
Decision-Making with Auto-Encoding Variational Bayes Romain Lopez Pierre Boyeau Nir Yosef Michael I. Jordan Jeffrey Regier BDL 434 10,591 0 17 Feb 2020
Momentum Improves Normalized SGD Ashok Cutkosky Harsh Mehta ODL 71 125 0 09 Feb 2020
Stein's Lemma for the Reparameterization Trick with Exponential Family Mixtures Wu Lin Mohammad Emtiyaz Khan Mark Schmidt 71 31 0 29 Oct 2019
Variational Bayes on Manifolds Minh-Ngoc Tran D. Nguyen Duy Nguyen 67 23 0 08 Aug 2019
Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations Wu Lin Mohammad Emtiyaz Khan Mark Schmidt BDL 49 71 0 07 Jun 2019
Practical Deep Learning with Bayesian Principles Kazuki Osawa S. Swaroop Anirudh Jain Runa Eschenhagen Richard Turner Rio Yokota Mohammad Emtiyaz Khan BDL UQCV 139 247 0 06 Jun 2019
High-dimensional copula variational approximation through transformation M. Smith Rubén Loaiza-Maya David J. Nott 67 33 0 16 Apr 2019
The Variational Predictive Natural Gradient Da Tang Rajesh Ranganath BDL DRL 32 10 0 07 Mar 2019
Provable Smoothness Guarantees for Black-Box Variational Inference Justin Domke 48 35 0 24 Jan 2019
Bayesian variational inference for exponential random graph models Linda S. L. Tan Nial Friel 60 17 0 10 Nov 2018
Variance reduction properties of the reparameterization trick Ming Xu M. Quiroz Robert Kohn Scott A. Sisson AAML 75 67 0 27 Sep 2018
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam Mohammad Emtiyaz Khan Didrik Nielsen Voot Tangkaratt Wu Lin Y. Gal Akash Srivastava ODL 149 272 0 13 Jun 2018
Bayesian Deep Net GLM and GLMM Minh-Ngoc Tran Nghia Nguyen David J. Nott Robert Kohn BDL 116 75 0 25 May 2018
Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Models Hugh Salimbeni Stefanos Eleftheriadis J. Hensman BDL 73 86 0 24 Mar 2018
Noisy Natural Gradient as Variational Inference Guodong Zhang Shengyang Sun David Duvenaud Roger C. Grosse ODL 72 212 0 06 Dec 2017
Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients Lukas Balles Philipp Hennig 75 169 0 22 May 2017
Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models Mohammad Emtiyaz Khan Wu Lin BDL 51 137 0 13 Mar 2017
Optimization Methods for Large-Scale Machine Learning Léon Bottou Frank E. Curtis J. Nocedal 246 3,216 0 15 Jun 2016
Faster Eigenvector Computation via Shift-and-Invert Preconditioning Dan Garber Laurent Dinh Chi Jin Jascha Narain Sohl-Dickstein Samy Bengio Praneeth Netrapalli Aaron Sidford 272 3,702 0 26 May 2016
Gaussian variational approximation with sparse precision matrices Linda S. L. Tan David J. Nott 102 77 0 18 May 2016
Overdispersed Black-Box Variational Inference Francisco J. R. Ruiz Michalis K. Titsias David M. Blei 125 47 0 03 Mar 2016
Automatic Differentiation Variational Inference A. Kucukelbir Dustin Tran Rajesh Ranganath Andrew Gelman David M. Blei 114 718 0 02 Mar 2016
Hierarchical Variational Models Rajesh Ranganath Dustin Tran David M. Blei DRL VLM 86 338 0 07 Nov 2015
Beyond Convexity: Stochastic Quasi-Convex Optimization Elad Hazan Kfir Y. Levy Shai Shalev-Shwartz 68 176 0 08 Jul 2015
Leave Pima Indians alone: binary regression as a benchmark for Bayesian computation Nicolas Chopin James Ridgway 69 76 0 29 Jun 2015
Variational Gaussian Copula Inference Shaobo Han X. Liao David B. Dunson Lawrence Carin 66 56 0 19 Jun 2015
Optimizing Neural Networks with Kronecker-factored Approximate Curvature James Martens Roger C. Grosse ODL 104 1,014 0 19 Mar 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.9K 150,115 0 22 Dec 2014
New insights and perspectives on the natural gradient method James Martens ODL 73 624 0 03 Dec 2014
Black Box Variational Inference Rajesh Ranganath S. Gerrish David M. Blei DRL BDL 142 1,167 0 31 Dec 2013
Inferring Parameters and Structure of Latent Variable Models by Variational Bayes H. Attias CML BDL 71 666 0 23 Jan 2013
ADADELTA: An Adaptive Learning Rate Method Matthew D. Zeiler ODL 155 6,625 0 22 Dec 2012
Stochastic Variational Inference Matt Hoffman David M. Blei Chong-Jun Wang John Paisley BDL 259 2,625 0 29 Jun 2012
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression Tim Salimans David A. Knowles 118 252 0 28 Jun 2012
Variational Bayesian Inference with Stochastic Search John Paisley David M. Blei Michael I. Jordan BDL 106 499 0 27 Jun 2012
Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations Linda S. L. Tan David J. Nott 109 49 0 17 May 2012