Accuracy of empirical projections of high-dimensional Gaussian matrices

Let $epsilon\in\R^{M\times M}$ be a centered Gaussian matrix whose entries are independent with variance $\sigma^2$. The accuracy of reduced-rank projections of $X=C+epsilon$ with a deterministic matrix $C$ is studied. It is shown that a combination of amplitude and shape of the singular value spectrum of $C$ is responsible for the quality of the empirical reduced-rank projection, which we quantify for some prototype matrices $C$. Our approach does not involve analytic perturbation theory of linear operators and covers the situation of multiple singular values in particular. The main proof relies on a bound on the supremum over some non-centered process with Bernstein tails which is built on a slicing of the Grassmann manifold along a geometric grid of concentric Hilbert-Schmidt norm balls. The results are accompanied by lower bounds under various assumptions.