Education Commons^™

Using Differential Item Functioning To Test For Inter-Rater Reliability In Constructed Response Items, Tamara Beth Miller

Theses and Dissertations

ABSTRACT

USING DIFFERENTIAL ITEM FUNCTIONING TO TEST FOR INTER-RATER RELIABILITY IN CONSTRUCTED RESPONSE ITEMS

by

Tamara B. Miller

The University of Wisconsin-Milwaukee, 2015

Under the Supervision of Professor Cindy M. Walker

This study used empirical and simulated data to compare and contrast traditional measures of inter-rater reliability to a novel measure of inter-rater scoring differences in constructed response items. The purpose of this research was to investigate an alternative measure of inter-rater differences, based in modern test theory, that does not require a fully crossed design for its calculation. The proposed, novel measure of inter-rater differences utilizes methods that are …

Dissecting The Impact Of Dif/Dbf On Ability Estimation And Person Fit, Kevin Cappaert

Theses and Dissertations

Prior research has shown that differential item functioning (DIF) and differential bundle functioning (DBF) can influence ability estimation in unidimensional item repose theory (IRT); however, the relationship between ability estimation and uniform and non-uniform DIF/DBF has not been thoroughly investigated. Therefore, a simulation study was conducted to more thoroughly investigate how DIF/DBF and other related factors influence ability estimation in IRT. The factors examined included bundle size, the sum of uniform DIF in a bundle, magnitude of non-uniform DIF in each item in a bundle, balance of reference and focal group examinees, test length, and impact. Results indicated that an …

Assessing Mathematical Competence In Second Language: Exploring Dif Evidences From Pisa Malaysian Data, Mazlina Husin

Theses and Dissertations

The year 2003 represents a significant milestone in the history of education development in Malaysia. From 2003, mathematics and science will be taught in English. This change in policy was deemed necessary to ensure that Malaysians are able to keep abreast with scientific and technological development that is mostly recorded in the English language. However, an unintended consequence of this language change was its huge impact on the national education system and the assessment of that system as well. Whenever students are not tested in their home language, one validity issue arises, which is how language, rather than the targeted …

Differential Item Functioning Analysis Of The Herrmann Brain Dominance Instrument, Jared Andrew Lees

Theses and Dissertations

Differential item functioning (DIF) is present when examinees who have the same level of a trait have a different probability of correctly answering a test item intended to measure that trait (Shepard & Averill, 1981). The following study is a DIF analysis of the Herrmann Brain Dominance Instrument (HBDI), a preference profiling instrument developed by Herrmann International to help individuals identify their dominant preferences and then classify their level of dominance into four preference quadrants. Examinees who completed the American English version of the instrument were classified as the reference group and examinees of the International English version were classified …