Engineering Commons^™

Using Uncertainty To Interpret Supervised Machine Learning Predictions, Michael C. Darling

Electrical and Computer Engineering ETDs

Traditionally, machine learning models are assessed using methods that estimate an average performance against samples drawn from a particular distribution. Examples include the use of cross-validation or hold0out to estimate classification error, F-score, precision, and recall.

While these measures provide valuable information, they do not tell us a model's certainty relative to particular regions of the input space. Typically there are regions where the model can differentiate the classes with certainty, and regions where the model is much less certain about its predictions.

In this dissertation we explore numerous approaches for quantifying uncertainty in the individual predictions made by supervised …