Exact and Stable Covariance Estimation from Quadratic Sampling via Convex Programming

Statistical inference and information processing of high-dimensional data streams and random processes often require efficient and accurate estimation of their second-order statistics. With rapidly changing data and limited storage, it is desirable to extract the covariance structure from a single pass over the data stream. In this paper, we explore a quadratic random sampling model which imposes minimal memory requirement and low computational complexity during the sampling process, and are shown to be optimal in preserving low-dimensional covariance structures. Specifically, two popular structural assumptions of covariance matrices, namely sparsity and low rank, are investigated. We show that a covariance matrix with either structure can be perfectly recovered from a minimal number of sub-Gaussian quadratic measurements, via efficient convex relaxation for respective structure. The proposed convex optimization algorithm has a variety of potential applications in large-scale data stream processing, high-frequency wireless communication, phase space tomography in optics, non-coherent subspace detection, etc. By introducing a novel notion of mixed-norm restricted isometry property (RIP-$\ell_{2}/\ell_{1}$), we show that our method admits accurate and universal recovery in the absence of noise, as soon as the number of measurements exceeds the theoretic sampling limits. We also show this approach is robust to noise and imperfect structural assumptions, i.e. it admits high-accuracy recovery even when the covariance matrix is only approximately low-rank or sparse. Our methods are inspired by the recent breakthroughs in phase retrieval, and the analysis framework herein recovers and improves upon best-known phase retrieval guarantees with simpler proofs.