A Fourier Analytical Approach to Estimation of Smooth Functions in Gaussian Shift Model

We study the estimation of $f(\btheta)$ under Gaussian shift model $\bx = \btheta+\bxi$, where $\btheta \in \RR^d$ is an unknown parameter, $\bxi \sim \mathcal{N}(\mathbf{0},\bSigma)$ is the random noise with covariance matrix $\bSigma$, and $f$ is a given function which belongs to certain Besov space with smoothness index $s>1$. Let $\sigma^2 = \|\bSigma\|_{op}$ be the operator norm of $\bSigma$ and $\sigma^{-2\alpha} = \br(\bSigma)$ be its effective rank with some $0<\alpha<1$ and $\sigma>0$. We develop a new estimator $g(\bx)$ based on a Fourier analytical approach that achieves effective bias reduction. We show that when the intrinsic dimension of the problem is large enough such that nontrivial bias reduction is needed, the mean square error (MSE) rate of $g(\bx)$ is $O\big(\sigma^2 \vee \sigma^{2(1-\alpha)s}\big)$ as $\sigma\rightarrow 0$. By developing new methods to establish the minimax lower bounds under standard Gaussian shift model, we show that this rate is indeed minimax optimal and so is $g(\bx)$. The minimax rate implies a sharp threshold on the smoothness $s$ such that for only $f$ with smoothness above the threshold, $f(\btheta)$ can be estimated efficiently with an MSE rate of the order $O(\sigma^2)$. Normal approximation and asymptotic efficiency were proved for $g(\bx)$ under mild restrictions. Furthermore, we propose a data-driven procedure to develop an adaptive estimator when the covariance matrix $\bSigma$ is unknown. Numerical simulations are presented to validate our analysis. The simplicity of implementation and its superiority over the plug-in approach indicate the new estimator can be applied to a broad range of real world applications.