Over the years, computer vision researchers have spent an immense amount of effort on designing image features for visual object recognition tasks. We propose to leverage this valuable experience to guide the task of training deep neural networks. Specifically, we propose a new unsupervised pretraining method for deep neural networks. Our idea is to pretrain the network through the task of replicating classic hand-designed features. By learning to replicate these features, the deep network integrates previous research knowledge and learns to model visual objects in a way similar to the hand-designed features. In the succeeding finetuning step, it further learns object-specific representations from labeled data and this boosts its classification power. We demonstrate the application of our method on the pedestrian detection task. We pretrain two deep convolutional neural networks where one replicates the histogram of oriented gradients feature, and the other replicates the region covariance feature. After finetuning, we achieve substantially better performance than those networks pretrained using existing methods.
View on arXiv