A vanilla Rao--Blackwellisation of Metropolis-Hastings algorithms

Casella and Robert (1996) presented a general Rao--Blackwellisation principle for accept-reject and Metropolis-Hastings schemes that leads to significant decreases in the variance of the resulting estimators, but at a high cost in computing and storage. Adopting a completely different perspective, we introduce instead a universal scheme that guarantees variance reductions in all Metropolis-Hastings based estimators while keeping the computing cost under control. We establish a central limit theorems for the improved estimators and illustrate their performances on toy examples.