Evaluating model calibration in classification

19 February 2019

Fredrik Lindsten

Papers citing "Evaluating model calibration in classification"

50 / 62 papers shown

Title
Performance Estimation in Binary Classification Using Calibrated Confidence Juhani Kivimäki Jakub Białek W. Kuberski J. Nurminen 58 0 0 08 May 2025
Similarity-Distance-Magnitude Universal Verification Allen Schmaltz UQCV AAML 265 0 0 27 Feb 2025
Optimizing Estimators of Squared Calibration Errors in Classification Sebastian G. Gruber Francis Bach 82 1 0 24 Feb 2025
Early Stopping in Contextual Bandits and Inferences Zihan Cui 59 0 0 05 Feb 2025
Rethinking Early Stopping: Refine, Then Calibrate Eugene Berta David Holzmüller Michael I. Jordan Francis Bach 75 0 0 31 Jan 2025
The Curious Case of Arbitrariness in Machine Learning Prakhar Ganesh Afaf Taik G. Farnadi 71 2 0 28 Jan 2025
Confidence Calibration of Classifiers with Many Classes Adrien LeCoz Stéphane Herbin Faouzi Adjed UQCV 44 1 0 05 Nov 2024
Calibrating Expressions of Certainty Peiqi Wang Barbara D. Lam Yingcheng Liu Ameneh Asgari-Targhi Yikang Shen W. Wells Tina Kapur Polina Golland 43 1 0 06 Oct 2024
ReliOcc: Towards Reliable Semantic Occupancy Prediction via Uncertainty Learning Song Wang Zhongdao Wang Jiawei Yu Wentong Li Bailan Feng Junbo Chen Jianke Zhu UQCV 46 3 0 26 Sep 2024
Reassessing How to Compare and Improve the Calibration of Machine Learning Models M. Chidambaram Rong Ge 74 1 0 06 Jun 2024
Calibration of Continual Learning Models Lanpei Li Elia Piccoli Andrea Cossu Davide Bacciu Vincenzo Lomonaco CLL 47 2 0 11 Apr 2024
AI, Meet Human: Learning Paradigms for Hybrid Decision Making Systems Clara Punzi Roberto Pellungrini Mattia Setzu F. Giannotti D. Pedreschi 27 5 0 09 Feb 2024
Calibration by Distribution Matching: Trainable Kernel Calibration Metrics Charles Marx Sofian Zalouk Stefano Ermon 32 8 0 31 Oct 2023
Set Learning for Accurate and Calibrated Models Lukas Muttenthaler Robert A. Vandermeulen Qiuyi Zhang Thomas Unterthiner Klaus-Robert Muller 41 2 0 05 Jul 2023
Minimum-Risk Recalibration of Classifiers Zeyu Sun Dogyoon Song Alfred Hero 43 5 0 18 May 2023
Measuring Classification Decision Certainty and Doubt Alexander M. Berenbeim Iain J. Cruickshank Susmit Jha Robert H. Thomson Nathaniel D. Bastian UQCV 15 4 0 25 Mar 2023
Uncertainty Calibration for Counterfactual Propensity Estimation in Recommendation Wenbo Hu Xin Sun Qiang liu Wenbo Hu Shu Wu 47 0 0 23 Mar 2023
Rethinking Confidence Calibration for Failure Prediction Fei Zhu Zhen Cheng Xu-Yao Zhang Cheng-Lin Liu UQCV 22 39 0 06 Mar 2023
On the Richness of Calibration Benedikt Höltgen Robert C. Williamson 18 9 0 08 Feb 2023
Calibrating AI Models for Wireless Communications via Conformal Prediction K. Cohen Sangwoo Park Osvaldo Simeone S. Shamai 37 6 0 15 Dec 2022
AdaFocal: Calibration-aware Adaptive Focal Loss Arindam Ghosh Thomas Schaaf Matthew R. Gormley FedML UQCV 37 26 0 21 Nov 2022
Stop Measuring Calibration When Humans Disagree Joris Baan Wilker Aziz Barbara Plank Raquel Fernández 29 53 0 28 Oct 2022
Calibration tests beyond classification David Widmann Fredrik Lindsten Dave Zachariah 35 17 0 21 Oct 2022
Class-wise and reduced calibration methods Michael Panchenko Anes Benmerzoug Miguel de Benito Delgado 23 0 0 07 Oct 2022
Variable-Based Calibration for Machine Learning Classifiers Mark Kelly Padhraic Smyth 29 4 0 30 Sep 2022
GRASP: A Goodness-of-Fit Test for Classification Learning Adel Javanmard M. Mehrabi 22 0 0 05 Sep 2022
Calibrated Selective Classification Adam Fisch Tommi Jaakkola Regina Barzilay 34 17 0 25 Aug 2022
Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networks Sérgio A. Balanya Juan Maroñas Daniel Ramos OOD 43 14 0 31 Jul 2022
Which models are innately best at uncertainty estimation? Ido Galil Mohammed Dabbah Ran El-Yaniv UQCV 34 5 0 05 Jun 2022
What is Your Metric Telling You? Evaluating Classifier Calibration under Context-Specific Definitions of Reliability John Kirchenbauer Jacob Oaks Eric Heim UQCV 41 4 0 23 May 2022
Metrics of calibration for probabilistic predictions Imanol Arrieta-Ibarra Paman Gujral Jonathan Tannen M. Tygert Cherie Xu 55 20 0 19 May 2022
Training Uncertainty-Aware Classifiers with Conformalized Deep Learning Bat-Sheva Einbinder Yaniv Romano Matteo Sesia Yanfei Zhou UQCV 34 49 0 12 May 2022
On the Usefulness of the Fit-on-the-Test View on Evaluating Calibration of Classifiers Markus Kängsepp Kaspar Valk Meelis Kull 35 3 0 16 Mar 2022
T-Cal: An optimal test for the calibration of predictive models Donghwan Lee Xinmeng Huang Hamed Hassani Yan Sun 27 20 0 03 Mar 2022
Calibrated Learning to Defer with One-vs-All Classifiers Rajeev Verma Eric Nalisnick 26 43 0 08 Feb 2022
Understanding Square Loss in Training Overparametrized Neural Network Classifiers Tianyang Hu Jun Wang Wei Cao Zhenguo Li UQCV AAML 48 19 0 07 Dec 2021
The Box Size Confidence Bias Harms Your Object Detector Johannes Gilg Torben Teepe Fabian Herzog Gerhard Rigoll ObjD 23 4 0 03 Dec 2021
Robustness and Reliability When Training With Noisy Labels Amanda Olmin Fredrik Lindsten OOD NoLa 24 14 0 07 Oct 2021
Can We Leverage Predictive Uncertainty to Detect Dataset Shift and Adversarial Examples in Android Malware Detection? Deqiang Li Tian Qiu Shuo Chen Qianmu Li Shouhuai Xu AAML 80 12 0 20 Sep 2021
A Survey of Uncertainty in Deep Neural Networks J. Gawlikowski Cedrique Rovile Njieutcheu Tassi Mohsin Ali Jongseo Lee Matthias Humt ... R. Roscher Muhammad Shahzad Wen Yang R. Bamler Xiaoxiang Zhu BDL UQCV OOD 66 1,115 0 07 Jul 2021
Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Error Ondrej Bohdal Yongxin Yang Timothy M. Hospedales UQCV OOD 52 21 0 17 Jun 2021
Revisiting the Calibration of Modern Neural Networks Matthias Minderer Josip Djolonga Rob Romijnders F. Hubis Xiaohua Zhai N. Houlsby Dustin Tran Mario Lucic UQCV 51 360 0 15 Jun 2021
Danish Fungi 2020 -- Not Just Another Image Recognition Dataset Lukás Picek Milan Šulc Jirí Matas J. Heilmann‐Clausen T. Jeppesen T. Læssøe T. Frøslev 30 54 0 18 Mar 2021
Distribution-free uncertainty quantification for classification under label shift Aleksandr Podkopaev Aaditya Ramdas UQCV 22 86 0 04 Mar 2021
Prostate Tissue Grading with Deep Quantum Measurement Ordinal Regression Santiago Toledo-Cortés Diego H. Useche Fabio A. González 11 0 0 04 Mar 2021
On Calibration and Out-of-domain Generalization Yoav Wald Amir Feder D. Greenfeld Uri Shalit OODD 30 153 0 20 Feb 2021
Combining Ensembles and Data Augmentation can Harm your Calibration Yeming Wen Ghassen Jerfel Rafael Muller Michael W. Dusenberry Jasper Snoek Balaji Lakshminarayanan Dustin Tran UQCV 36 63 0 19 Oct 2020
A Generic Methodology for the Statistically Uniform & Comparable Evaluation of Automated Trading Platform Components A. Sokolovsky Luca Arnaboldi 23 2 0 21 Sep 2020
Calibration of Model Uncertainty for Dropout Variational Inference M. Laves Sontje Ihler Karl-Philipp Kortmann T. Ortmaier BDL UQCV 32 18 0 20 Jun 2020
Distribution-free binary classification: prediction sets, confidence intervals and calibration Chirag Gupta Aleksandr Podkopaev Aaditya Ramdas UQCV 33 79 0 18 Jun 2020