DEUP: Direct Epistemic Uncertainty Prediction

Epistemic Uncertainty (EU) is the part of out-of-sample prediction error due to the lack of knowledge of the learner. While existing work focuses on model variance as a proxy of EU, we propose a principled framework for directly estimating it by learning to predict the generalization error and subtracting an estimate of aleatoric uncertainty, i.e., intrinsic unpredictability, using a secondary learner. We discuss the merits of this novel interpretation of EU, and highlight how it differs from variance-based proxies of EU and addresses their shortcomings. Our framework, Direct Epistemic Uncertainty Prediction (DEUP) is particularly interesting in interactive learning environments, where the learner is allowed to acquire novel examples in each round. Through a wide set of experiments, we illustrate how existing methods in sequential model optimization can be improved with EU estimates from DEUP, and how DEUP can be used to drive exploration in reinforcement learning. We also evaluate the quality of uncertainty estimates from DEUP for probabilistic image classification and predicting synergies of drug combinations.