Social and Behavioral Sciences Commons^™

Detecting Careless Cases In Practice Tests, Steven Nydick

Chinese/English Journal of Educational Measurement and Evaluation | 教育测量与评估双语期刊

In this paper, we present a novel method for detecting careless responses in a low-stakes practice exam using machine learning models. Rather than classifying test-taker responses as careless based on model fit statistics or knowledge of truth, we built a model to predict significant changes in test scores between a practice test and an official test based on attributes of practice test items. We extracted features from practice test items using hypotheses about how careless test takers respond to items and cross-validated model performance to optimize out-of-sample predictions and reduce heteroscedasticity when predicting the closest official test. All analyses use …