BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation

Recent advanced approaches to semantic segmentation in general rely on deep convolutional networks trained with annotated segmentation masks. Such pixel-level supervision demands expensive labor, and thus limits the performance of deep networks that desire plenty of training data. In this paper, we make use of bounding box annotations to supervise convolutional networks for semantic segmentation. From these boxes, we estimate segmentation masks with the help of region proposals. These masks are used to update the convolutional network, which is in turn fed back to mask estimation. This procedure is iterated. This method, called "BoxSup", is thoroughly evaluated on the PASCAL VOC 2012 semantic segmentation benchmark. Our method with only bounding boxes as supervision shows results competitive with previous methods supervised by masks. Our method supervised by a few masks and a large amount of bounding boxes yields a new state-of-the-art result (71.0% mean IoU) on this challenging dataset.