Social and Behavioral Sciences Commons^™

Rubric Rating With Mfrm Vs. Randomly Distributed Comparative Judgment: A Comparison Of Two Approaches To Second-Language Writing Assessment, Maureen Estelle Sims

Theses and Dissertations

The purpose of this study is to explore a potentially more practical approach to direct writing assessment using computer algorithms. Traditional rubric rating (RR) is a common yet highly resource-intensive evaluation practice when performed reliably. This study compared the traditional rubric model of ESL writing assessment and many-facet Rasch modeling (MFRM) to comparative judgment (CJ), the new approach, which shows promising results in terms of reliability and validity. We employed two groups of raters”novice and experienced”and used essays that had been previously double-rated, analyzed with MFRM, and selected with fit statistics. We compared the results of the novice and experienced …