Computer Engineering Commons^™

Information-Theoretic Model Diagnostics (Infomod), Armin Esmaeilzadeh

UNLV Theses, Dissertations, Professional Papers, and Capstones

Model validation is a critical step in the development, deployment, and governance of machine learning models. During the validation process, the predictive power of a model is measured on unseen datasets with a variety of metrics such as Accuracy and F1-Scores for classification tasks. Although the most used metrics are easy to implement and understand, they are aggregate measures over all the segments of heterogeneous datasets, and therefore, they do not identify the performance variation of a model among different data segments. The lack of insight into how the model performs over segments of unseen datasets has raised significant challenges …