A study of the classification of low-dimensional data with supervised manifold learning

Supervised manifold learning methods learn data representations by preserving the geometric structure of data while enhancing the separation between data samples from different classes. In this paper, we propose a theoretical study of supervised manifold learning for classification. We first focus on the supervised Laplacian eigenmaps algorithm and study the conditions under which this method computes low-dimensional embeddings where different classes become linearly separable. We then consider arbitrary supervised manifold learning algorithms that compute a linearly separable embedding and study the accuracy of the classifiers given by the out-of-sample extensions of these embeddings. We characterize the classification accuracy in terms of several parameters of the classifier such as the separation between different classes in the embedding, the regularity of the interpolation function and the number of training samples. The proposed analysis is supported by experiments on synthetic and real data and has potential for guiding the design of classifiers for intrinsically low-dimensional data.