COVID-19 detection using chest X-rays: is lung segmentation important for generalization?

We evaluated the generalization capability of deep neural networks (DNNs), trained to classify chest X-rays as COVID-19, normal or pneumonia, using a relatively small and mixed dataset. We proposed a DNN to perform lung segmentation and classification, stacking a segmentation module (U-Net), an original intermediate module and a classification module (DenseNet201). To evaluate generalization, we tested the DNN with an external dataset (from distinct localities) and used Bayesian inference to estimate probability distributions of performance metrics. Our DNN achieved 0.917 AUC on the external test dataset, and a DenseNet without segmentation, 0.906. Bayesian inference indicated mean accuracy of 76.1% and [0.695, 0.826] 95% HDI (high density interval, which concentrates 95% of the metric's probability mass) with segmentation and, without segmentation, 71.7% and [0.646, 0.786]. We proposed a novel DNN evaluation technique, using Layer-wise Relevance Propagation (LRP) and Brixia scores. LRP heatmaps indicated that areas where radiologists found strong COVID-19 symptoms and attributed high Brixia scores are the most important for the stacked DNN classification. External validation showed smaller accuracies than internal, indicating difficulty in generalization, which segmentation improves. Performance in the external dataset and LRP analysis suggest that DNNs can be trained in small and mixed datasets and detect COVID-19.