Responsible model deployment via model-agnostic uncertainty learning.

Reliably predicting potential failure risks of machine learning (ML) systems when deployed with production data is a crucial aspect of trustworthy AI. This paper introduces the , a novel post-hoc for estimating failure risks and predictive uncertainties of black-box classification model. In addition to providing a , the decomposes the uncertainty estimates into aleatoric and epistemic uncertainty components, thus giving informative insights into the sources of uncertainty inducing the failures.