On Many-to-Many Mapping Between Concordance Correlation Coefficient and Mean Square Error

The concordance correlation coefficient (CCC) is one of the most widely used reproducibility indices, introduced by Lin in 1989. In addition to its extensive use in assay validation, CCC serves various other roles. It is most commonly used as a metric to quantify an inter-rater agreement. It is also popular as a performance metric for the time-series and multivariate prediction problems. Despite being a popular performance metric, there has been hardly any attempt to design a loss function tailor-made to train the corresponding predictive deep-learning models. While minimisation of $L_p$ norm of the errors (e.g., mean square error/MSE minimisation) aims at CCC maximisation effectively, we establish in this paper the sheer ineffectiveness of this popular strategy, with concrete reasons. For the very first time, we present the formulation for many-to-many mapping existing between MSE and CCC. We also establish conditions for CCC optimisation when given a fixed MSE; and then as a logical next step, when given a fixed set of error coefficients. We present a few mathematical paradoxes as well (albeit apparent ones) that we discovered through this CCC reformulation and optimisation endeavour. We propose the loss function to be $\left(\frac{MSE(x,y)}{cov(x,y)}\right)^{2n}, n\in\mathbb{N}$.