We propose the worst-case treatment effect (WTE) across all subpopulations of a given size, a conservative notion of topline treatment effect. Compared to the average treatment effect (ATE), whose validity relies on the covariate distribution of collected data, WTE is robust to unanticipated covariate shifts, and positive findings guarantee uniformly valid treatment effects over subpopulations. We develop a semiparametrically efficient estimator for the WTE, leveraging machine learning-based estimates of the heterogeneous treatment effect and propensity score. By virtue of satisfying a key (Neyman) orthogonality property, our estimator enjoys central limit behavior---oracle rates with true nuisance parameters---even when estimates of nuisance parameters converge at slower rates. For both randomized trials and observational studies, we establish a semiparametric efficiency bound, proving that our estimator achieves the optimal asymptotic variance. On real datasets where robustness to covariate shift is of core concern, we illustrate the non-robustness of ATE under even mild distributional shift, and demonstrate that the WTE guards against brittle findings that are invalidated by unanticipated covariate shifts.
View on arXiv