Free energy methods for efficient exploration of mixture posterior densities

Because of their multimodality, mixture posterior densities are difficult to sample with standard Markov chain Monte Carlo (MCMC) methods. We propose a strategy to enhance the sampling of MCMC in this context, using a biasing procedure which originates from computational statistical physics. The principle is first to choose a "reaction coordinate", that is, a direction in which the target density is multimodal. In a second step, the marginal log-density of the reaction coordinate is estimated; this quantity is called "free energy" in the computational statistical physics literature. To this end, we use adaptive biasing Markov chain algorithms which adapt their invariant distribution on the fly, in order to overcome sampling barriers along the chosen reaction coordinate. Finally, we perform an importance sampling step in order to remove the bias and recover the true posterior. A crucial point is the choice of the reaction coordinate. We show that a convenient and efficient reaction coordinate is the hyper-parameter that determines the order of magnitude of the variance of each component. We also show how to adapt the method to perform model choice between different number of components. We illustrate our approach by analyzing two real data sets.