59
65

Solving kk-center Clustering (with Outliers) in MapReduce and Streaming, almost as Accurately as Sequentially

Abstract

Center-based clustering is a fundamental primitive for data analysis and becomes very challenging for large datasets. In this paper, we focus on the popular kk-center variant which, given a set SS of points from some metric space and a parameter k<Sk<|S|, requires to identify a subset of kk centers in SS minimizing the maximum distance of any point of SS from its closest center. A more general formulation, introduced to deal with noisy datasets, features a further parameter zz and allows up to zz points of SS (outliers) to be disregarded when computing the maximum distance from the centers. We present coreset-based 2-round MapReduce algorithms for the above two formulations of the problem, and a 1-pass Streaming algorithm for the case with outliers. For any fixed ϵ>0\epsilon>0, the algorithms yield solutions whose approximation ratios are a mere additive term ϵ\epsilon away from those achievable by the best known polynomial-time sequential algorithms, a result that substantially improves upon the state of the art. Our algorithms are rather simple and adapt to the intrinsic complexity of the dataset, captured by the doubling dimension DD of the metric space. Specifically, our analysis shows that the algorithms become very space-efficient for the important case of small (constant) DD. These theoretical results are complemented with a set of experiments on real-world and synthetic datasets of up to over a billion points, which show that our algorithms yield better quality solutions over the state of the art while featuring excellent scalability, and that they also lend themselves to sequential implementations much faster than existing ones.

View on arXiv
Comments on this paper