Bayesian and minimax estimators of loss

We study the problem of loss estimation that involves for an observable $X \sim f_{\theta}$ the choice of a first-stage estimator $\hat{\gamma}$ of $\gamma(\theta)$, incurred loss $L=L(\theta, \hat{\gamma})$, and the choice of a second-stage estimator $\hat{L}$ of $L$. We consider both: (i) a sequential version where the first-stage estimate and loss are fixed and optimization is performed at the second-stage level, and (ii) a simultaneous version with a Rukhin-type loss function designed for the evaluation of $(\hat{\gamma}, \hat{L})$ as an estimator of $(\gamma, L)$. We explore various Bayesian solutions and provide minimax estimators for both situations (i) and (ii). The analysis is carried out for several probability models, including multivariate normal models $N_d(\theta, \sigma^2 I_d)$ with both known and unknown $\sigma^2$, Gamma, univariate and multivariate Poisson, and negative binomial models, and relates to different choices of the first-stage and second-stage losses. The minimax findings are achieved by identifying least favourable of sequence of priors and depend critically on particular Bayesian solution properties, namely situations where the second-stage estimator $\hat{L}(x)$ is constant as a function of $x$.