Data augmentation for models based on rejection sampling

We present a data augmentation scheme to perform Markov chain Monte Carlo inference for models where data generation involves a rejection sampling algorithm. Our idea is to instantiate the rejected proposals preceding each data point, and we show that this can be done easily and efficiently. The resulting joint probability over observed and rejected variables can be much simpler than the marginal distribution over the observed variables, which often involve intractable integrals. Our algorithm is an instance of a growing body of work on exact Markov chain Monte Carlo inference for doubly-intractable distributions and we consider two such problems. The first is a Bayesian analysis of the matrix Langevin distribution on the Stiefel manifold, and the second Bayesian inference for a nonparametric Gaussian process density model. Our experiments demonstrate superior performance over state-of-the-art sampling algorithms for such problems.