On the Surprising Efficiency of Committee-based Models

Committee-based models, i.e., model ensembles or cascades, are underexplored in recent work on developing efficient models. While committee-based models themselves are not new, there lacks a systematic understanding of their efficiency in comparison with single models. To fill this gap, we conduct a comprehensive analysis of the efficiency of committee-based models and find that committee-based models provide a complementary paradigm to achieve superior efficiency without tuning the architecture: a simple ensemble or cascade of existing networks can be considerably more efficient than state-of-the-art single models, even outperforming sophisticated neural architecture search methods. The superior efficiency of committee-based models holds true for several tasks, including image classification, video classification, and semantic segmentation, and various architecture families, such as EfficientNet, ResNet, MobileNetV2, and X3D.