Analyzing Cost-Sensitive Surrogate Losses via $\mathcal{H}$-calibration

This paper aims to understand whether machine learning models should be trained using cost-sensitive surrogates or cost-agnostic ones (e.g., cross-entropy). Analyzing this question through the lens of $\mathcal{H}$-calibration, we find that cost-sensitive surrogates can strictly outperform their cost-agnostic counterparts when learning small models under common distributional assumptions. Since these distributional assumptions are hard to verify in practice, we also show that cost-sensitive surrogates consistently outperform cost-agnostic surrogates on classification datasets from the UCI repository. Together, these make a strong case for using cost-sensitive surrogates in practice.

@article{shah2025_2502.19522,
  title={ Analyzing Cost-Sensitive Surrogate Losses via $\mathcal{H}$-calibration },
  author={ Sanket Shah and Milind Tambe and Jessie Finocchiaro },
  journal={arXiv preprint arXiv:2502.19522},
  year={ 2025 }
}