Statistical Models Commons^™

Model Evaluation Based On The Distribution Of Estimated Absolute Prediction Error, Lu Tian, Tianxi Cai, Els Goetghebeur, L. J. Wei

Harvard University Biostatistics Working Paper Series

The construction of a reliable, practically useful prediction rule for future response is heavily dependent on the "adequacy" of the fitted regression model. In this article, we consider the absolute prediction error, the expected value of the absolute difference between the future and predicted responses, as the model evaluation criterion. This prediction error is easier to interpret than the average squared error and is equivalent to the mis-classification error for the binary outcome. We show that the distributions of the apparent error and its cross-validation counterparts are approximately normal even under a misspecified fitted model. When the prediction rule is …