Deep Manifold Transformation for Dimension Reduction

Nonlinear dimensionality reduction (NLDR) plays an important role in feature extraction and visualization of high dimensional data. It transforms patterns of interest, or manifolds, in data to regions in a lower-dimensional latent space. When the manifolds are highly complicated, the NLDR transformation has to provide sufficient nonlinearity for downstream tasks. This paper proposes a novel method, called {\em deep manifold transformation} (DMT), to tackle this problem. As a multi-layer neural network, DMT can support a higher degree of nonlinearity than single-layer methods. Continuation strategy is applied during training to address the local minimum problem common in manifold learning algorithms. Such a learned DMT network can generalize to unseen data whereas traditional manifold learning methods only provide an embedding of the training data. Extensive experiments, comparisons, and ablation studies demonstrate that DMT can deliver results superior to UMAP and t-SNE, and other leading manifold-based NLDR methods.