Open Access. Powered by Scholars. Published by Universities.®

Education Commons

Open Access. Powered by Scholars. Published by Universities.®

Series

Educational Assessment, Evaluation, and Research

English Publications

Automated writing evaluation

Publication Year

Articles 1 - 4 of 4

Full-Text Articles in Education

Automated Writing Evaluation For Formative Assessment Of Second Language Writing: Investigating The Accuracy And Usefulness Of Feedback As Part Of Argument-Based Validation, Jim Ranalli, Stephanie Link, Evgeny Chukharev-Hudilainen Feb 2016

Automated Writing Evaluation For Formative Assessment Of Second Language Writing: Investigating The Accuracy And Usefulness Of Feedback As Part Of Argument-Based Validation, Jim Ranalli, Stephanie Link, Evgeny Chukharev-Hudilainen

English Publications

An increasing number of studies on the use of tools for automated writing evaluation (AWE) in writing classrooms suggest growing interest in their potential for formative assessment. As with all assessments, these applications should be validated in terms of their intended interpretations and uses. A recent argument-based validation framework outlined inferences that require backing to support integration of one AWE tool, Criterion, into a college-level English as a Second Language (ESL) writing course. The present research appraised evidence for the assumptions underlying two inferences in this argument. In the first of two studies, we assessed evidence for the evaluation inference ...


Discourse Classification Into Rhetorical Functions For Awe Feedback, Elena Cotos, Nick Pendar Jan 2016

Discourse Classification Into Rhetorical Functions For Awe Feedback, Elena Cotos, Nick Pendar

English Publications

This paper reports on the development of the analysis engine for the Research Writing Tutor (RWT), an AWE program designed to provide genre and discipline-specific feedback on the functional units of research article discourse. Unlike traditional NLP-based applications that categorize complete documents, RWT’s analyzer categorizes every sentence in the text as both a communicative move and a rhetorical step. We describe the construction of a cascade of two support vector machine classifiers trained on a multi-disciplinary corpus of annotated Introduction texts. This work not only demonstrates the usefulness of NLP for automated genre analysis, but also paves the road ...


Validity Arguments For Diagnostic Assessment Using Automated Writing Evaluation, Carol Chapelle, Elena Cotos, Jooyoung Lee Jan 2015

Validity Arguments For Diagnostic Assessment Using Automated Writing Evaluation, Carol Chapelle, Elena Cotos, Jooyoung Lee

English Publications

Two examples demonstrate an argument-based approach to validation of diagnostic assessment using automated writing evaluation (AWE). Criterion ®, was developed by Educational Testing Service to analyze students’ papers grammatically, providing sentence-level error feedback. An interpretive argument was developed for its use as part of the diagnostic assessment process in undergraduate university English for academic purposes (EAP) classes. The Intelligent Academic Discourse Evaluator (IADE) was developed for use in graduate EAP university classes, where the goal was to help students improve their discipline-specific writing. The validation for each was designed to support claims about the intended purposes of the assessments. We present ...


Furthering And Applying Move/Step Constructs: Technology-Driven Marshalling Of Swalesian Genre Theory For Eap Pedagogy, Elena Cotos, Sarah Huffman, Stephanie Link Jan 2015

Furthering And Applying Move/Step Constructs: Technology-Driven Marshalling Of Swalesian Genre Theory For Eap Pedagogy, Elena Cotos, Sarah Huffman, Stephanie Link

English Publications

John Swales' seminal work has inspired a wealth of research with important pedagogical implications for genre-based writing instruction. Continuing the prolific move analysis tradition in EAP research, this article presents empirically devised and validated cross-disciplinary IMRD move/step frameworks for the research article genre and demonstrates how Swales' move and step concepts underlying these frameworks formed the foundation of innovative genre-based automated writing evaluation technology. Overall, this paper makes the relationships between genre theory, genre analysis, and genre instruction explicit, demonstrating that move analysis is a powerful and promising theoretical, analytic, and teaching construct. With that, we take Swales' vision ...