Learning compact, interpretable image representations is a very natural task which has not been solved satisfactorily even for simple classes of binary images. In this paper, we review various ways of composing parts (or experts) for binary data and argue that competitive forms of interaction are best suited to learn low-dimensional representations. We propose a new composition rule which discourages parts from focusing on similar structures and which penalizes opposing votes strongly so that abstaining from voting becomes more attractive. We also introduce a novel sequential initialization procedure based on a process of oversimplification and correction. Experiments show that with our approach very intuitive models can be learned.
View on arXiv