Business Commons^™

Validity And Reliability Of Situational Judgement Test Scores: A New Approach Based On Cognitive Diagnosis Models, Miguel A. Sorrel, Julio Olea, Francisco José Abad, Jimmy De La Torre, David Aguado, Filip Lievens

Research Collection Lee Kong Chian School Of Business

Conventional methods for assessing the validity and reliability of situational judgment test (SJT) scores have proven to be inadequate. For example, factor analysis techniques typically lead to nonsensical solutions, and assumptions underlying Cronbach's alpha coefficient are violated due to the multidimensional nature of SJTs. In the current article, we describe how cognitive diagnosis models (CDMs) provide a new approach that not only overcomes these limitations but that also offers extra advantages for scoring and better understanding SJTs. The analysis of the Q-matrix specification, model fit, and model parameter estimates provide a greater wealth of information than traditional procedures do. Our …