Measuring dependence powerfully and equitably

For high-dimensional datasets, it is common to evaluate a measure of dependence on every variable pair and retain the highest-scoring pairs for follow-up. If the statistic used systematically assigns higher scores to some relationship types over others, important relationships may be overlooked. This difficulty is avoided if the statistic is equitable [Reshef et al. 2015a], i.e., if, for some measure of noise, it assigns similar scores to equally noisy relationships regardless of relationship type. In this paper, we introduce and characterize a population measure of dependence called MIC*. We show three ways that MIC* can be viewed: as the population value of MIC, a highly equitable statistic from [Reshef et al. 2011], as a canonical "smoothing" of mutual information, and as the supremum of an infinite sequence defined in terms of optimal one-dimensional partitions of the marginals of the joint distribution. Based on this theory, we introduce an efficient algorithm for computing MIC* from the density of a pair of random variables, and we define a new consistent estimator MICe for MIC* that is efficiently computable. (In contrast, there is no known polynomial-time algorithm for computing MIC.) We show through simulations that MICe has better bias-variance properties than MIC, and that it has high equitability with respect to R^2 on a set of functional relationships. While MICe is designed for equitability rather than independence testing, we introduce a related statistic, TICe, that is a trivial side-product of the computation of MICe. We prove the consistency of independence testing based on TICe and show in simulations that this approach achieves excellent power. This paper is accompanied by a companion paper [Reshef et al. 2015b] focused on in-depth empirical evaluation of several leading measures of dependence that finds that the performance of MICe and TICe is state-of-the-art.