Education Commons^™

The Effect Of Raters And Rating Conditions On The Reliability Of The Missionary Teaching Assessment, Abigail Christine Ure

Theses and Dissertations

This study investigated how 2 different rating conditions, the controlled rating condition (CRC) and the uncontrolled rating condition (URC), effected rater behavior and the reliability of a performance assessment (PA) known as the Missionary Teaching Assessment (MTA). The CRC gives raters the capability to manipulate (pause, rewind, fast-forward) video recordings of an examinee's performance as they rate while the URC does not give them this capability (i.e., the rater must watch the recording straight through without making any manipulations). Few studies have compared the effect of these two rating conditions on ratings. Ryan et al. (1995) analyzed the impact of …

Psychometric Properties Of Postsecondary Students' Course Evaluations, Michael J. Drysdale

All Graduate Theses and Dissertations, Spring 1920 to Summer 2023

Several experts in the area of postsecondary student evaluations of courses have concluded that they are stable or reliable measures as well as being measures that provide ways of making valid inferences regarding teacher effectiveness. Often these experts have offered these conclusions without supporting evidence. Surprisingly, a thorough review of the literature revealed very few reported test-retest reliability studies of course evaluations and the results from these studies are contradictory. In the area of validity, the conclusions offered by scholars who conducted meta-analyses of mutlisection course studies are inconsistent. This leads to the following two research questions:

1. What is …

Investigating The Reliability And Construct Validity Of A Measure Of Preservice Teachers' Self-Efficacy For Tpack, Nicolette Michelle Smith

Theses and Dissertations

The TPACK framework is becoming increasingly pervasive in teacher education. Researchers and practitioners have been seeking reliable and valid ways to measure the constructs associated with the TPACK framework. This study describes the results of both an item review and the reliability and construct validity investigation of the scores from an instrument measuring self-efficacy for the constructs in the TPACK framework. Content-matter experts and the literature were used in order to perform the item review, while both an exploratory and a confirmatory factor analysis were performed in order to assess construct validity. Cronbach's alpha and Raykov's rho were used to …

Estimating The Reliability Of Concept Map Ratings Using A Scoring Rubric Based On Three Attributes, Laura Jimenez

Theses and Dissertations

Concept maps provide a way to assess how well students have developed an organized understanding of how the concepts taught in a unit are interrelated and fit together. However, concept maps are challenging to score because of the idiosyncratic ways in which students organize their knowledge (McClure, Sonak, & Suen, 1999). The construct a map or C-mapping" task has been shown to capture students' organized understanding. This "C-mapping" task involves giving students a list of concepts and asking them to produce a map showing how these concepts are interrelated. The purpose of this study was twofold: (a) to determine to …

Reliability Of Pre-Service Physical Education Teachers' Coding Of Teaching Videos Using Studiocode Analysis Software, Keven Prusak, Brigham Dye, Charles R. Graham, Susan Graser

Faculty Publications

This study examines the coding reliability and accuracy of pre-service teachers in a teaching methods class using digital video (DV)-based teaching episodes and Studiocode analysis software. Student self-analysis of DV footage may offer a high tech solution to common shortfalls of traditional systematic observation and reflection practices by increasing the amount, timeliness, and accuracy of performance feedback. What is yet to be determined is whether students can reliably and accurately analyze such footage. Using Studiocode software, student analyses were compared to those of experts to determine coding reliability and content accuracy. The results of this study indicate that with less …