Simulation-based Regularized Logistic Regression

We develop an omnibus framework for regularized logistic regression by simulation-based inference, exploiting two important results on scale mixtures of normals. By carefully choosing a hierarchical model for the likelihood by one type of mixture, and how regularization may be implemented by another, we obtain subtly different MCMC schemes with varying efficiency depending on the data type (binary v. binomial, say) and the desired estimator (maximum likelihood, maximum a posteriori, posterior mean, etc.). Advantages of this umbrella approach include flexibility, computational efficiency, application in p >> n settings, uncertainty estimates, variable selection, and an ability to assess the optimal degree of regularization. We compare aspects of our proposed methods to modern alternatives on synthetic and real data. An R package called reglogit is available on CRAN.