Optimal designs for Lasso and Dantzig selector using Expander Codes

In this article, we investigate the high-dimensional regression problem when the design matrix is the normalized adjacency matrix of a unbalanced expander graph. These bipartite graphs encode a family of error-correcting codes, namely expander codes, that are asymptotically good codes which can be both encoded and decoded from a constant fraction of errors in polynomial time. Using these matrices as design matrices, we prove that the L2-prediction error and the L1-risk of the lasso and the Dantzig selector are optimal up to an explicit multiplicative constant. Thus we can estimate a high-dimensional vector with an error term similar to the one given by an oracle estimator (i.e. when one knows the support of the largest coordinates, in absolute value, of the signal in advance). Interestingly, we show that these design matrices have an explicit restricted eigenvalue. Equivalently, they satisfy the restricted eigenvalue assumption and the compatibility condition with an explicit constant. In the last part, we use the recent construction of unbalanced expander graphs due to Guruswami, Umans, and Vadhan, to provide a deterministic construction of these design matrices.