Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift

6 June 2019

Balaji Lakshminarayanan

Jasper Snoek

UQCV

ArXiv PDF HTML

Papers citing "Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift"

50 / 1,042 papers shown

Title
Combining Ensembles and Data Augmentation can Harm your Calibration Yeming Wen Ghassen Jerfel Rafael Muller Michael W. Dusenberry Jasper Snoek Balaji Lakshminarayanan Dustin Tran UQCV 32 63 0 19 Oct 2020
Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian Inference Disi Ji Padhraic Smyth M. Steyvers 39 45 0 19 Oct 2020
Stationary Activations for Uncertainty Calibration in Deep Learning Lassi Meronen Christabella Irwanto Arno Solin UQCV BDL 14 18 0 19 Oct 2020
Active Domain Adaptation via Clustering Uncertainty-weighted Embeddings Viraj Prabhu Arjun Chandrasekaran Kate Saenko Judy Hoffman OOD 106 124 0 16 Oct 2020
Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness Long Zhao Ting Liu Xi Peng Dimitris N. Metaxas OOD AAML 27 165 0 15 Oct 2020
Exploring the Uncertainty Properties of Neural Networks' Implicit Priors in the Infinite-Width Limit Ben Adlam Jaehoon Lee Lechao Xiao Jeffrey Pennington Jasper Snoek UQCV BDL 31 15 0 14 Oct 2020
Ensemble Distillation for Structured Prediction: Calibrated, Accurate, Fast-Choose Three Steven Reich David Mueller Matthew Wiesner BDL OOD UQCV 30 13 0 13 Oct 2020
Learning Calibrated Uncertainties for Domain Shift: A Distributionally Robust Learning Approach Haoxu Wang Zhiding Yu Yisong Yue Anima Anandkumar Anqi Liu Junchi Yan OOD UQCV 13 4 0 08 Oct 2020
Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures Benjamin Kompa Jasper Snoek Andrew L. Beam UQCV BDL 34 29 0 06 Oct 2020
Learnable Uncertainty under Laplace Approximations Agustinus Kristiadi Matthias Hein Philipp Hennig UQCV BDL 22 30 0 06 Oct 2020
An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence Agustinus Kristiadi Matthias Hein Philipp Hennig BDL 26 9 0 06 Oct 2020
Explaining The Efficacy of Counterfactually Augmented Data Divyansh Kaushik Amrith Rajagopal Setlur Eduard H. Hovy Zachary Chase Lipton CML 28 81 0 05 Oct 2020
Neural Bootstrapper Minsuk Shin Hyungjoon Cho Hyun-Seok Min Sungbin Lim UQCV BDL 22 7 0 02 Oct 2020
Ask-n-Learn: Active Learning via Reliable Gradient Representations for Image Classification Bindya Venkatesh Jayaraman J. Thiagarajan 11 4 0 30 Sep 2020
Why have a Unified Predictive Uncertainty? Disentangling it using Deep Split Ensembles U. Sarawgi W. Zulfikar Rishab Khincha Pattie Maes PER UQCV BDL UD 21 7 0 25 Sep 2020
A Unifying Review of Deep and Shallow Anomaly Detection Lukas Ruff Jacob R. Kauffmann Robert A. Vandermeulen G. Montavon Wojciech Samek Marius Kloft Thomas G. Dietterich Klaus-Robert Muller UQCV 25 782 0 24 Sep 2020
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics Swabha Swayamdipta Roy Schwartz Nicholas Lourie Yizhong Wang Hannaneh Hajishirzi Noah A. Smith Yejin Choi 51 429 0 22 Sep 2020
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika D. Song Jacob Steinhardt ELM RALM 78 3,969 0 07 Sep 2020
Stochastic-YOLO: Efficient Probabilistic Object Detection under Dataset Shifts Tiago Azevedo R. D. Jong Matthew Mattina Partha P. Maji UQCV 18 15 0 07 Sep 2020
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning Martin Mundt Yongjun Hong Iuliia Pliushch Visvanathan Ramesh CLL 30 146 0 03 Sep 2020
Estimating the Brittleness of AI: Safety Integrity Levels and the Need for Testing Out-Of-Distribution Performance A. Lohn 27 12 0 02 Sep 2020
Webly Supervised Image Classification with Self-Contained Confidence Jingkang Yang Xue Jiang Weirong Chen Xiaopeng Yan Huabin Zheng Ping Luo Wayne Zhang 28 15 0 27 Aug 2020
Surrogate Model For Field Optimization Using Beta-VAE Based Regression Ajitabh Kumar 25 0 0 26 Aug 2020
Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarks Arber Zela Julien N. Siems Lucas Zimmer Jovita Lukasik M. Keuper Frank Hutter 42 76 0 22 Aug 2020
Privacy Preserving Recalibration under Domain Shift Rachel Luo Shengjia Zhao Jiaming Song Jonathan Kuck Stefano Ermon Silvio Savarese 14 3 0 21 Aug 2020
A Survey on Assessing the Generalization Envelope of Deep Neural Networks: Predictive Uncertainty, Out-of-distribution and Adversarial Samples Julia Lust Alexandru Paul Condurache UQCV AAML AI4CE 29 8 0 21 Aug 2020
Hey Human, If your Facial Emotions are Uncertain, You Should Use Bayesian Neural Networks! Maryam Matin Matias Valdenegro-Toro UQCV BDL CVBM 32 2 0 17 Aug 2020
Beyond Point Estimate: Inferring Ensemble Prediction Variation from Neuron Activation Strength in Recommender Systems Zhe Chen Yuyan Wang Dong Lin D. Cheng Lichan Hong Ed H. Chi Claire Cui 31 16 0 17 Aug 2020
Survey of XAI in digital pathology Milda Pocevičiūtė Gabriel Eilertsen Claes Lundström 14 56 0 14 Aug 2020
Unbiased Learning for the Causal Effect of Recommendation Masahiro Sato S. Takemori Janmajay Singh Tomoko Ohkuma CML OffRL 9 68 0 11 Aug 2020
Deep Bayesian Bandits: Exploring in Online Personalized Recommendations Dalin Guo S. Ktena Ferenc Huszár Pranay K. Myana Wenzhe Shi Alykhan Tejani OffRL 33 40 0 03 Aug 2020
Cold Posteriors and Aleatoric Uncertainty Ben Adlam Jasper Snoek Samuel L. Smith BDL UQCV 28 23 0 31 Jul 2020
Generative Classifiers as a Basis for Trustworthy Image Classification Radek Mackowiak Lynton Ardizzone Ullrich Kothe Carsten Rother 22 4 0 29 Jul 2020
Bayesian Few-Shot Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes Jake C. Snell R. Zemel 33 63 0 20 Jul 2020
Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles Tárik S. Salem H. Langseth H. Ramampiaro UQCV 21 36 0 19 Jul 2020
Probabilistic Neighbourhood Component Analysis: Sample Efficient Uncertainty Estimation in Deep Learning Ankur Mallick Chaitanya Dwivedi B. Kailkhura Gauri Joshi T. Y. Han UQCV BDL 42 6 0 18 Jul 2020
On Robustness and Transferability of Convolutional Neural Networks Josip Djolonga Jessica Yung Michael Tschannen Rob Romijnders Lucas Beyer ... D. Moldovan Sylvain Gelly N. Houlsby Xiaohua Zhai Mario Lucic OOD 13 154 0 16 Jul 2020
Transferable Calibration with Lower Bias and Variance in Domain Adaptation Ximei Wang Mingsheng Long Jianmin Wang Michael I. Jordan 25 52 0 16 Jul 2020
Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics V. Ramasesh Ethan Dyer M. Raghu CLL 24 174 0 14 Jul 2020
Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users Laurent Valentin Jospin Wray Buntine F. Boussaïd Hamid Laga Bennamoun OOD BDL UQCV 29 611 0 14 Jul 2020
On uncertainty estimation in active learning for image segmentation Bo Li T. S. Alstrøm UQCV 28 14 0 13 Jul 2020
Bayesian Deep Ensembles via the Neural Tangent Kernel Bobby He Balaji Lakshminarayanan Yee Whye Teh BDL UQCV 14 116 0 11 Jul 2020
Machine Learning Explainability for External Stakeholders Umang Bhatt Mckane Andrus Adrian Weller Alice Xiang FaML SILM 16 58 0 10 Jul 2020
Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks Shreyas Padhy Zachary Nado Jie Jessie Ren J. Liu Jasper Snoek Balaji Lakshminarayanan UQCV 17 45 0 10 Jul 2020
Boundary thickness and robustness in learning models Yaoqing Yang Rekha Khanna Yaodong Yu A. Gholami Kurt Keutzer Joseph E. Gonzalez Kannan Ramchandran Michael W. Mahoney OOD 18 37 0 09 Jul 2020
URSABench: Comprehensive Benchmarking of Approximate Bayesian Inference Methods for Deep Neural Networks Meet P. Vadera Adam D. Cobb B. Jalaeian Benjamin M. Marlin BDL UQCV 27 16 0 08 Jul 2020
Diverse Ensembles Improve Calibration Asa Cooper Stickland Iain Murray UQCV FedML 27 26 0 08 Jul 2020
Deep Ensemble Analysis for Imaging X-ray Polarimetry A. L. Peirson R. Romani H. Marshall J. Steiner L. Baldini 13 19 0 08 Jul 2020
Counterfactual Data Augmentation using Locally Factored Dynamics Silviu Pitis Elliot Creager Animesh Garg BDL OffRL 26 85 0 06 Jul 2020
Increasing Trustworthiness of Deep Neural Networks via Accuracy Monitoring Zhihui Shao Jianyi Yang Shaolei Ren HILM 12 10 0 03 Jul 2020