Positive definite matrices and the Symmetric Stein Divergence

Positive definite matrices arise in a dazzling variety of applications. They enjoy this ubiquity perhaps due to their rich geometric structure. In particular, positive definite matrices form a convex cone whose strict interior is also a differentiable Riemannian manifold. Building on the conic and manifold views, we advocate the \emph{Symmetric Stein Divergence} (S-Divergence) as a `natural' distance-like function on positive matrices. We motivate its naturalness in a sequence of results that connect it to the Riemannian metric on positive matrices. Going beyond, we show that the S-Divergence has many interesting properties of its own: most notably, its square-root turns out to be a metric. We discuss some properties of this metric, including Hilbert space embeddability, before concluding the paper with a list of open problems. We hope that our paper encourages others to further study the S-Divergence and its applications.