Education Commons^™

The Reliability Of Single-Item Assessments, Sijun Zhang

Legacy Theses & Dissertations (2009 - 2024)

Single-item assessments have become more popular recently in distinct areas, even though there is no consensus about whether they are sufficiently reliable. Researchers have developed methods to estimate the reliability of single-item assessments, some are based on factor analysis (method FA), correction for attenuation (method CA), and others employ Molenaar and Sijtsma’s theory (method MS), coefficient λ6 (method λ6), or latent class model (method LCRC). However, no empirical study has investigated which method estimates the reliability of single-item assessment most precisely. This study investigated this question via a simulation study. To represent assessments as found in practice, the simulation study …

Using Generalizability Theory To Investigate Sources Of Variance Of The Autism Diagnostic Observation Schedule-2 With Trainees, Dorothy Rita Parriott

UNLV Theses, Dissertations, Professional Papers, and Capstones

Autism Spectrum Disorder (ASD) can be considered a serious developmental concern, which, complicated by its rising rate, creates a challenge for psychologists in properly and consistently diagnosing the disorder. Various types of assessments such as reporting measures, observation systems, and standardized assessments are currently used in the identification of ASD. Any one instrument typically examines multiple domains of functioning such as intellectual, neuropsychological (including adaptive, attention, sensory, motor, language, memory, executive functioning, academic, and social/emotional), and behavior (social and repetitive/restricted). Often, evaluators combine assessments that were not originally meant to detect ASD with those that were intended for that purpose. …

The Effect Of Raters And Rating Conditions On The Reliability Of The Missionary Teaching Assessment, Abigail Christine Ure

Theses and Dissertations

This study investigated how 2 different rating conditions, the controlled rating condition (CRC) and the uncontrolled rating condition (URC), effected rater behavior and the reliability of a performance assessment (PA) known as the Missionary Teaching Assessment (MTA). The CRC gives raters the capability to manipulate (pause, rewind, fast-forward) video recordings of an examinee's performance as they rate while the URC does not give them this capability (i.e., the rater must watch the recording straight through without making any manipulations). Few studies have compared the effect of these two rating conditions on ratings. Ryan et al. (1995) analyzed the impact of …

Psychometric Properties Of Postsecondary Students' Course Evaluations, Michael J. Drysdale

All Graduate Theses and Dissertations, Spring 1920 to Summer 2023

Several experts in the area of postsecondary student evaluations of courses have concluded that they are stable or reliable measures as well as being measures that provide ways of making valid inferences regarding teacher effectiveness. Often these experts have offered these conclusions without supporting evidence. Surprisingly, a thorough review of the literature revealed very few reported test-retest reliability studies of course evaluations and the results from these studies are contradictory. In the area of validity, the conclusions offered by scholars who conducted meta-analyses of mutlisection course studies are inconsistent. This leads to the following two research questions:

1. What is …

Investigating The Reliability And Construct Validity Of A Measure Of Preservice Teachers' Self-Efficacy For Tpack, Nicolette Michelle Smith

Theses and Dissertations

The TPACK framework is becoming increasingly pervasive in teacher education. Researchers and practitioners have been seeking reliable and valid ways to measure the constructs associated with the TPACK framework. This study describes the results of both an item review and the reliability and construct validity investigation of the scores from an instrument measuring self-efficacy for the constructs in the TPACK framework. Content-matter experts and the literature were used in order to perform the item review, while both an exploratory and a confirmatory factor analysis were performed in order to assess construct validity. Cronbach's alpha and Raykov's rho were used to …

Estimating The Reliability Of Concept Map Ratings Using A Scoring Rubric Based On Three Attributes, Laura Jimenez

Theses and Dissertations

Concept maps provide a way to assess how well students have developed an organized understanding of how the concepts taught in a unit are interrelated and fit together. However, concept maps are challenging to score because of the idiosyncratic ways in which students organize their knowledge (McClure, Sonak, & Suen, 1999). The construct a map or C-mapping" task has been shown to capture students' organized understanding. This "C-mapping" task involves giving students a list of concepts and asking them to produce a map showing how these concepts are interrelated. The purpose of this study was twofold: (a) to determine to …

Reliability Of Pre-Service Teachers Coding Of Teaching Videos Using Video-Annotation Tools, Brigham R. Dye

Theses and Dissertations

Teacher education programs that aspire to helping pre-service teachers develop expertise must help students engage in deliberate practice along dimensions of teaching expertise. However, field teaching experiences often lack the quantity and quality of feedback that is needed to help students engage in meaningful teaching practice. The limited availability of supervising teachers makes it difficult to personally observe and evaluate each student teacher's field teaching performances. Furthermore, when a supervising teacher debriefs such an observation, the supervising teacher and student may struggle to communicate meaningfully about the teaching performance. This is because the student teacher and supervisor often have very …