Estimator selection in the Gaussian setting

We consider the problem of estimating the mean $f$ of a Gaussian vector $Y$ the components of which are independent with a common variance that we assume to be unknown. Our estimation procedure is based on estimator selection. More precisely, we start with a collection $\FF$ of estimators of $f$ based on $Y$ and, with the same data $Y$, we aim at selecting an estimator among $\FF$ with the smallest Euclidean risk. We allow the cardinality of $\FF$ to be very large (possibly infinite) and also the dependency of the estimators with respect to the data to be possibly unknown. We establish a non-asymptotic risk bound for the selected estimator. When $\FF$ consists of linear estimators, we derive from this bound an oracle-type inequality. For illustration, we carry out two simulation studies. One aims at comparing our procedure to cross-validation for choosing a tuning parameter. The other shows how to implement our approach to solve the problem of variable selection in practice.