The Wilson Machine for Image Modeling

Learning the distribution of natural images is one of the hardest problems in machine learning. We break down this challenging problem by mapping images into a hierarchy of binary images (bit-planes). In this representation, the top bit-plane is critical, having fluctuations in structures over a vast range of scales. The ones below go through a gradual stochastic heating process to disorder. We turn this representation into a directed probabilistic graphical model, transforming the learning problem into the unsupervised learning of the distribution of the critical bit-plane and the supervised learning of the conditional distributions for the remaining bit-planes. We learnt the conditional distributions by logistic regression in a convolutional architecture. Conditioned on the critical binary image, this simple architecture can generate large natural-looking images with many shades of gray, without the use of hidden units.