18
13

kkFolden: kk-Fold Ensemble for Out-Of-Distribution Detection

Abstract

Out-of-Distribution (OOD) detection is an important problem in natural language processing (NLP). In this work, we propose a simple yet effective framework kkFolden, which mimics the behaviors of OOD detection during training without the use of any external data. For a task with kk training labels, kkFolden induces kk sub-models, each of which is trained on a subset with k1k-1 categories with the left category masked unknown to the sub-model. Exposing an unknown label to the sub-model during training, the model is encouraged to learn to equally attribute the probability to the seen k1k-1 labels for the unknown label, enabling this framework to simultaneously resolve in- and out-distribution examples in a natural way via OOD simulations. Taking text classification as an archetype, we develop benchmarks for OOD detection using existing text classification datasets. By conducting comprehensive comparisons and analyses on the developed benchmarks, we demonstrate the superiority of kkFolden against current methods in terms of improving OOD detection performances while maintaining improved in-domain classification accuracy. The code and datasets can be found at: \url{https://github.com/ShannonAI/kfolden-ood-detection}.

View on arXiv
Comments on this paper