65
138

Black-box αα-divergence Minimization

Abstract

Black-box alpha (BB-α\alpha) is a new approximate inference method based on the minimization of α\alpha-divergences. BB-α\alpha scales to large datasets because it can be implemented using stochastic gradient descent. BB-α\alpha can be applied to complex probabilistic models with little effort since it only requires as input the likelihood function and its gradients. These gradients can be easily obtained using automatic differentiation. By changing the divergence parameter α\alpha, the method is able to interpolate between variational Bayes (VB) (α0\alpha \rightarrow 0) and an algorithm similar to expectation propagation (EP) (α=1\alpha = 1). Experiments on probit regression and neural network regression and classification problems show that BB-α\alpha with non-standard settings of α\alpha, such as α=0.5\alpha = 0.5, usually produces better predictions than with α0\alpha \rightarrow 0 (VB) or α=1\alpha = 1 (EP).

View on arXiv
Comments on this paper

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.