Data Augmentation in Emotion Classification using Generative Adversarial Networks

It is a difficult task to classify images with multiple labels only using a small number of labeled samples and especially, with unbalanced distribution. In this paper we propose a data augmentation method using generative adversarial networks(GAN), that can complement and complete the data manifold, assist the classifier better to find margins or hyper-planes of neighboring classes, and finally lead to a better performance in emotion classification task. Specifically, we design a pipeline containing a CNN model as classifier and a cycle-consistent adversarial networks(CycleGAN) to generate supplementary data from given classes. In order to avoid gradient vanishing, we apply a least-squared distance in least squares generative adversarial networks(LSGAN) to adversarial loss. We also propose several evaluation methods on three benchmark facial epression datasets to validate GAN's contribution in data augmentation. Qualitative evaluation indicates that data manifolds show a significant improvement in distribution integrity and margin clarity between classes. Quantitative comparisons show that we can obtain 5%~10% increase in the classification accuracy after employing our data augmentation technique.