Open Access. Powered by Scholars. Published by Universities.®

Education Commons™

Open Access. Powered by Scholars. Published by Universities.®

2016

Inter-rater reliability

Articles 1 - 2 of 2

Full-Text Articles in Education

An Investigation Into Native And Non Native English Speaking Instructors' Assessment Of University Esl Student's Oral Presentation, Rubaiyat Jabeen Jan 2016

An Investigation Into Native And Non Native English Speaking Instructors' Assessment Of University Esl Student's Oral Presentation, Rubaiyat Jabeen

All Graduate Theses, Dissertations, and Other Capstone Projects

This thesis aims to conduct a mix-methods investigation into native English-speaking teachers (NESTs) and non-native English-speaking teachers (NNESTs) assessment of university level English as a second language (ESL) student's oral presentation. To collect data for this study, all faculty members and instructors currently teaching in at the departments of English and Communication Studies at Minnesota State University, Mankato (MNSU) were invited to participate in an online survey using Qualtrics Survey Software (Qualtrics). After receiving email invitations, altogether 31 people provided their consent to participate. Among the 31 teacher-participants, there were 19 NESTs and 12 NNESTs. The participants then took part …

Go to article

Inter-Rater Variability As Mutual Disagreement: Identifying Raters’ Divergent Points Of View, A. Gingerich, Susan E. Ramlo Dec 2015

Inter-Rater Variability As Mutual Disagreement: Identifying Raters’ Divergent Points Of View, A. Gingerich, Susan E. Ramlo

Susan E Ramlo

Whenever multiple observers provide ratings, even of the same performance,
inter-rater variation is prevalent. The resulting ‘idiosyncratic rater variance’ is considered
to be unusable error of measurement in psychometric models and is a threat to the
defensibility of our assessments. Prior studies of inter-rater variation in clinical assessments
have used open response formats to gather raters’ comments and justifications. This
design choice allows participants to use idiosyncratic response styles that could result in a
distorted representation of the underlying rater cognition and skew subsequent analyses. In
this study we explored rater variability using the structured response format of Q
methodology. …

Go to article