Finite sample posterior concentration in high-dimensional regression

We study the behavior of the posterior distribution in ultra high-dimensional Bayesian Gaussian linear regression models having $p\gg n$, with $p$ the number of predictors and $n$ the sample size. In particular, our focus is on obtaining non-asymptotic probabilistic bounds on the posterior probability assigned in neighborhoods of the true regression coefficient vector, $\beta^0$, with these bounds used to study contraction of the posterior. We assume that $\beta^0$ is approximately $S$-sparse and obtain universal bounds via a Schwartz-type argument, though only well-structured priors exhibit good properties. Based upon these finite sample bounds, we examine the implied asymptotic contraction rates for several examples showing that sparsely-structured and heavy-tail shrinkage priors exhibit rapid contraction rates. Using brute force, we also demonstrate that a stronger result holds for the Uniform-Gaussian prior, which indicates that our main result can be strengthened and reinforces the fact that the estimates of the denominator in the Schwartz-type arguments are not sharp in the finite sample regime.