An Observation on Lloyd's k-Means Algorithm in High Dimensions

Clustering and estimating cluster means are core problems in statistics and machine learning, with k-means and Expectation Maximization (EM) being two widely used algorithms. In this work, we provide a theoretical explanation for the failure of k-means in high-dimensional settings with high noise and limited sample sizes, using a simple Gaussian Mixture Model (GMM). We identify regimes where, with high probability, almost every partition of the data becomes a fixed point of the k-means algorithm. This study is motivated by challenges in the analysis of more complex cases, such as masked GMMs, and those arising from applications in Cryo-Electron Microscopy.

@article{silva-sánchez2025_2506.14952,
  title={ An Observation on Lloyd's k-Means Algorithm in High Dimensions },
  author={ David Silva-Sánchez and Roy R. Lederman },
  journal={arXiv preprint arXiv:2506.14952},
  year={ 2025 }
}