Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift

6 June 2019

Balaji Lakshminarayanan

Jasper Snoek

UQCV

ArXiv PDF HTML

Papers citing "Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift"

50 / 1,043 papers shown

Title
Assurance Monitoring of Learning Enabled Cyber-Physical Systems Using Inductive Conformal Prediction based on Distance Learning Dimitrios Boursinos X. Koutsoukos 48 11 0 07 Oct 2021
Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation Dennis Ulmer Christian Hardmeier J. Frellsen BDL UQCV UD EDL PER 55 48 0 06 Oct 2021
Deep Classifiers with Label Noise Modeling and Distance Awareness Vincent Fortuin Mark Collier F. Wenzel J. Allingham J. Liu Dustin Tran Balaji Lakshminarayanan Jesse Berent Rodolphe Jenatton E. Kokiopoulou UQCV 39 11 0 06 Oct 2021
Post-hoc Models for Performance Estimation of Machine Learning Inference Xuechen Zhang Samet Oymak Jiasi Chen UQCV 23 4 0 06 Oct 2021
$Δ$ -UQ: Accurate Uncertainty Quantification via Anchor Marginalization Rushil Anirudh Jayaraman J. Thiagarajan 38 1 0 05 Oct 2021
Dropout Q-Functions for Doubly Efficient Reinforcement Learning Takuya Hiraoka Takahisa Imagawa Taisei Hashimoto Takashi Onishi Yoshimasa Tsuruoka 13 105 0 05 Oct 2021
Synergizing between Self-Training and Adversarial Learning for Domain Adaptive Object Detection Muhammad Akhtar Munir M. H. Khan M. Sarfraz Mohsen Ali ObjD 17 53 0 01 Oct 2021
On the Importance of Gradients for Detecting Distributional Shifts in the Wild Rui Huang Andrew Geng Yixuan Li 197 332 0 01 Oct 2021
Offline Reinforcement Learning with Reverse Model-based Imagination Jianhao Wang Wenzhe Li Haozhe Jiang Guangxiang Zhu Siyuan Li Chongjie Zhang OffRL 117 61 0 01 Oct 2021
Out-of-Distribution Detection for Medical Applications: Guidelines for Practical Evaluation Karina Zadorozhny P. Thoral Paul Elbers Giovanni Cina OODD OOD 32 12 0 30 Sep 2021
Word-level confidence estimation for RNN transducers Mingqiu Wang H. Soltau Laurent El Shafey Izhak Shafran UQCV 26 5 0 28 Sep 2021
Unsolved Problems in ML Safety Dan Hendrycks Nicholas Carlini John Schulman Jacob Steinhardt 186 278 0 28 Sep 2021
A Step Towards Efficient Evaluation of Complex Perception Tasks in Simulation Jonathan Sadeghi Blaine Rogers James Gunn Thomas Saunders Sina Samangooei P. Dokania John Redford 41 8 0 28 Sep 2021
Training on Test Data with Bayesian Adaptation for Covariate Shift Aurick Zhou Sergey Levine OOD TTA 55 13 0 27 Sep 2021
A Physics inspired Functional Operator for Model Uncertainty Quantification in the RKHS Rishabh Singh José C. Príncipe 25 4 0 22 Sep 2021
Can We Leverage Predictive Uncertainty to Detect Dataset Shift and Adversarial Examples in Android Malware Detection? Deqiang Li Tian Qiu Shuo Chen Qianmu Li Shouhuai Xu AAML 80 12 0 20 Sep 2021
CompilerGym: Robust, Performant Compiler Optimization Environments for AI Research Chris Cummins Bram Wasti Jiadong Guo Brandon Cui Jason Ansel ... Jia-Wei Liu O. Teytaud Benoit Steiner Yuandong Tian Hugh Leather 35 69 0 17 Sep 2021
Reliable Neural Networks for Regression Uncertainty Estimation Tony Tohme Kevin Vanslette K. Youcef-Toumi UQCV BDL 29 15 0 16 Sep 2021
Deploying clinical machine learning? Consider the following... Charles Lu Kenglun Chang Praveer Singh S. Pomerantz S. Doyle Sujay S Kakarmath Christopher P. Bridge Jayashree Kalpathy-Cramer 54 4 0 14 Sep 2021
Robust Contrastive Active Learning with Feature-guided Query Strategies R. Krishnan Nilesh A. Ahuja Alok Sinha Mahesh Subedar Omesh Tickoo Ravi Iyer 26 1 0 13 Sep 2021
Mitigating Sampling Bias and Improving Robustness in Active Learning R. Krishnan Alok Sinha Nilesh A. Ahuja Mahesh Subedar Omesh Tickoo R. Iyer 13 9 0 13 Sep 2021
On the Impact of Spurious Correlation for Out-of-distribution Detection Yifei Ming Hang Yin Yixuan Li OODD 156 74 0 12 Sep 2021
A framework for benchmarking uncertainty in deep regression F. Schmähling Jörg Martin Clemens Elster UQCV 40 8 0 10 Sep 2021
Detecting and Mitigating Test-time Failure Risks via Model-agnostic Uncertainty Learning Preethi Lahoti Krishna P. Gummadi Gerhard Weikum 34 3 0 09 Sep 2021
Uncertainty Measures in Neural Belief Tracking and the Effects on Dialogue Policy Performance Carel van Niekerk A. Malinin Christian Geishauser Michael Heck Hsien-chin Lin Nurul Lubis Shutong Feng Milica Gavsić 26 9 0 09 Sep 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 71 697 0 04 Sep 2021
SHIFT15M: Fashion-specific dataset for set-to-set matching with several distribution shifts Masanari Kimura Takuma Nakamura Yuki Saito OOD 39 3 0 30 Aug 2021
Uncertainty-Aware Model Adaptation for Unsupervised Cross-Domain Object Detection Minjie Cai Minyi Luo Xionghu Zhong Hao Chen OOD 32 6 0 28 Aug 2021
Leveraging Uncertainty for Improved Static Malware Detection Under Extreme False Positive Constraints A. Nguyen Edward Raff Charles K. Nicholas James Holt 41 21 0 09 Aug 2021
Robust Semantic Segmentation with Superpixel-Mix Gianni Franchi Nacim Belkhir Mai Lan Ha Yufei Hu Andrei Bursuc V. Blanz Angela Yao UQCV 40 22 0 02 Aug 2021
Sequential Multivariate Change Detection with Calibrated and Memoryless False Detection Rates Oliver Cobb A. V. Looveren Janis Klaise 37 6 0 02 Aug 2021
Soft Calibration Objectives for Neural Networks A. Karandikar Nicholas Cain Dustin Tran Balaji Lakshminarayanan Jonathon Shlens Michael C. Mozer Becca Roelofs UQCV 25 85 0 30 Jul 2021
When Deep Learners Change Their Mind: Learning Dynamics for Active Learning Javad Zolfaghari Bengar Bogdan Raducanu Joost van de Weijer 19 10 0 30 Jul 2021
Are Bayesian neural networks intrinsically good at out-of-distribution detection? Christian Henning Francesco DÁngelo Benjamin Grewe UQCV BDL 31 10 0 26 Jul 2021
An Uncertainty-Aware Deep Learning Framework for Defect Detection in Casting Products Maryam Habibpour Hassan Gharoun AmirReza Tajally Afshar Shamsi Jokandan Hamzeh Asgharnezhad Abbas Khosravi S. Nahavandi UQCV 32 13 0 24 Jul 2021
Estimating Predictive Uncertainty Under Program Data Distribution Shift Yufei Li Simin Chen Wei Yang UQCV 24 3 0 23 Jul 2021
AUGCO: Augmentation Consistency-guided Self-training for Source-free Domain Adaptive Semantic Segmentation Viraj Prabhu Shivam Khare Deeksha Kartik Judy Hoffman TTA 31 21 0 21 Jul 2021
Iterative Distillation for Better Uncertainty Estimates in Multitask Emotion Recognition Didan Deng Liang Wu Bertram E. Shi 46 32 0 21 Jul 2021
High Frequency EEG Artifact Detection with Uncertainty via Early Exit Paradigm Lorena Qendro Alexander Campbell Pietro Lio Cecilia Mascolo 17 6 0 21 Jul 2021
Uncertainty Estimation and Out-of-Distribution Detection for Counterfactual Explanations: Pitfalls and Solutions Eoin Delaney Derek Greene Mark T. Keane 38 24 0 20 Jul 2021
Epistemic Neural Networks Ian Osband Zheng Wen M. Asghari Vikranth Dwaracherla M. Ibrahimi Xiyuan Lu Benjamin Van Roy UQCV BDL 32 99 0 19 Jul 2021
On the Importance of Regularisation & Auxiliary Information in OOD Detection John Mitros Brian Mac Namee 21 2 0 15 Jul 2021
Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks A. Malinin Neil Band Ganshin Alexander German Chesnokov ... Roginskiy Denis Mariya Shmatova Panos Tigas Boris Yangel UQCV OOD 29 128 0 15 Jul 2021
RBUE: A ReLU-Based Uncertainty Estimation Method of Deep Neural Networks Yufeng Xia Jun Zhang Zhiqiang Gong Tingsong Jiang W. Yao UQCV 25 1 0 15 Jul 2021
Uncertainty-Aware Reliable Text Classification Yibo Hu Latifur Khan EDL UQCV 40 33 0 15 Jul 2021
Federated Mixture of Experts M. Reisser Christos Louizos E. Gavves Max Welling FedML 36 25 0 14 Jul 2021
What classifiers know what they don't? Mohamed Ishmael Belghazi David Lopez-Paz 25 6 0 13 Jul 2021
Project Achoo: A Practical Model and Application for COVID-19 Detection from Recordings of Breath, Voice, and Cough Alexander Ponomarchuk I. Burenko Elian Malkin Ivan Nazarov V. Kokh Manvel Avetisian L. Zhukov 39 40 0 12 Jul 2021
Multi-headed Neural Ensemble Search Ashwin Raaghav Narayanan Arber Zela Tonmoy Saikia Thomas Brox Frank Hutter UQCV 34 4 0 09 Jul 2021
Differentially private training of neural networks with Langevin dynamics for calibrated predictive uncertainty Moritz Knolle Alexander Ziller Dmitrii Usynin R. Braren Marcus R. Makowski Daniel Rueckert Georgios Kaissis UQCV 52 2 0 09 Jul 2021