Lesion2Vec: Deep Metric Learning for Few Shot Multiple Lesions Recognition in Wireless Capsule Endoscopy

In this work, we present a unique approach for multiple lesion recognition in Wireless Capsule Endoscopy (WCE) video data. Few-shot Learning (FSL) aims to identify new concepts from only a small number of examples. Just as large amount of ground truth data may not be easily obtained for each class in a facial recognition nor biometric identification task, we leverage similar concept to develop a lesion recognition model based on Deep Metric Learning (DML) using Convolutional Siamese Neural Network (CSNN). We learned an embedding mapping function for each lesion category from only a few examples and then applied the learnt embedding mapping to identify lesions on a larger WCE video dataset. We demonstrated the efficacy of our method on real patient capsule endoscopy data and we bench-marked the performance with standard baseline classification models. We also showed that this approach can generalize to additional categories that the model never saw during training, obviating the need for fine-tuning the model parameters.