Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 2 of 2
Full-Text Articles in Education
A Qualitative Analysis Of Student Writing Rejected By An Automated Essay Scoring System, Nathanael Reinertsen, Toby Newton, Caithlin Power
A Qualitative Analysis Of Student Writing Rejected By An Automated Essay Scoring System, Nathanael Reinertsen, Toby Newton, Caithlin Power
Nathanael Reinertsen
The Australian Council for Educational Research’s online writing assessment, eWrite, uses Vantage Learning’s Intellimetric automated essay scoring (AES) system to score students’ writing. Approximately 5% of submissions are returned by the AES system unmarked and tagged with an error code, and are subsequently marked by a trained human rater. The raters provide numeric scores using an analytic rubric, and in some instances also provide written commentary about the scores.
This paper reports on a qualitative analysis of the commentaries of a sample of 86 scripts from 84 writers that were rejected by the AES and marked by trained human …
Why Can’T It Mark This One? A Qualitative Analysis Of Student Writing Rejected By An Automated Essay Scoring System, Nathanael Reinertsen
Why Can’T It Mark This One? A Qualitative Analysis Of Student Writing Rejected By An Automated Essay Scoring System, Nathanael Reinertsen
Nathanael Reinertsen
The difference in how humans read and how Automatic Essay Scoring (AES) systems process written language leads to a situation where a portion of student responses will be comprehensible to human markers, while being unable to be parsed by AES systems. This paper examines a number of pieces of student writing that were marked by trained human markers, but subsequently rejected by an AES system during the development of a scoring model for the eWrite online writing assessment that is offered by The Australian Council for Educational Research. The features of these ‘unscoreable’ responses are examined through a qualitative analysis. …