Education Commons^™

Evaluating Classifiers During Dataset Shift, Corey Fritsch

Theses and Dissertations

Deployment of a classifier into a machine learning application likely begins with training different types of algorithms on a subset of the available historical data and then evaluating them on datasets that are drawn from identical distributions. The goal of this evaluation process is to select the classifier that is believed to be most robust in maintaining good future performance, and then deploy that classifier to end-users who use it to make predictions on new data. Often times, predictive models are deployed in conditions that differ from those used in training, meaning that dataset shift occurred. In these situations, there …

Investigation Into The Measurement Of Student Ability: Refinements, Improvements And Enhanced Understanding Of Factors That Influence Performance, David Schreurs

Theses and Dissertations

The following series of experiments were all designed and implemented with the goal of understanding and improving undergraduate students understanding of chemistry. In order to accurately access changes in student’s chemistry understanding, the first chapter focuses on how a student’s chemistry proficiency can be most effectively measured. This includes practical recommendations for the average instructor as well as advanced methods more appropriate for researchers who require highly precise scores. With a better understanding on how to measure student proficiency, the next step was working to improve student understanding through various types of feedback following assessments. On top of establishing a …

Determining Predictor Importance In Multilevel Models For Longitudinal Data: An Extension Of Dominance Analysis, Luciana Pacheco Cancado

Theses and Dissertations

Longitudinal models are used not only to analyze the change of an outcome over time but also to describe what person-level and time-varying factors might influence this trend. Whenever a researcher is interested in the factors or predictors impacting an outcome, a common follow-up question asked is that of the relative importance of such factors. Hence, this study aimed to extend and evaluate Dominance Analysis (DA), a method used to determine the relative importance of predictors in various linear models (Budescu, 1993; Azen & Budescu, 2003; Azen, 2013), for use with longitudinal multilevel models. A simulation study was conducted to …

A Preliminary Evaluation Of An Academic Support Program, James Colin Bumby

Theses and Dissertations

The responsibility for graduating students is a school responsibility. A large urban Midwestern research university created the Nursing Endeavor Program (NEP) to ensure that students from underserved minorities, first generation, and/or low income are enabled to graduate successfully. There is a 100% retention rate for the period from admission to the nursing major in the junior year; however, in the pre-nursing years, the retention rate is only 50%. The role transition from pre-nursing student to student nurse can be challenging and stressful. A qualitative phenomenographic research approach was utilized to identify the factors facilitating and the factors hindering this transition. …

Evaluating Item Selection Methods For Adaptive Tests With Complex Content Constraints, Logan Rome

Theses and Dissertations

Adaptive testing designs have become go-to methods for large-scale test administration due to their ability to provide more accurate scores with fewer items. In recent years, new designs have been introduced, such as on-the-fly multistage testing (OMST), that combine the advantages of the well-established computerized adaptive testing (CAT) and multistage testing (MST) designs. While adaptive testing has attracted a tremendous amount of research, most studies have used only one set of test specifications to constrain the content of the test. Through Monte Carlo simulation, this study evaluated the effectiveness of CAT, MST, and OMST under varying levels of test specification …

School Culture And Climate For Younger Learners: Measurement And Association With Academic Achievement, Leon Joseph Gilman

Theses and Dissertations

This study seeks to understand the measurement of younger students’ perceptions of the school learning environment and their possible association with academic achievement. The target population is 4th and 5th grade students. Their perception of the school environment was compared to 7th graders by factor analysis, measurement invariance, differential item functioning, and hierarchical linear modeling. This study found that younger students’ perceptions are different from middle school students. However, like their middle school peers, these perceptions still predict academic performance.

Assessing Model-Data Fit For Compensatory And Non-Compensatory Multidimensional Item Response Models Using Vuong And Clarke Statistics, Leanne Freeman

Theses and Dissertations

The primary importance of the study was that no statistics were able to provide probabilistic statements about model-data fit between the non-nested compensatory and non-compensatory MIRT models. Secondarily, the Vuong and Clarke statistics have been utilized prolifically in economics and political science and have great potential to contribute to educational measurement. Finally, application of the Vuong and Clarke statistics will not only reduce the damage of misspecified MIRT models but will also promote the use of less known models, in this case, the non-compensatory models.

The purpose of the study was to investigate whether the Vuong and Clarke statistics can …

Making Test Batteries Adaptive By Using Multistage Testing Techniques, Wen Zeng

Theses and Dissertations

The objective of this dissertation research is to investigate the possibility to improve both reliability and validity for test batteries under the framework of multi-stage testing (MST). Two test battery designs that incorporate MST components were proposed and evaluated, one is a multistage test battery (MSTB) design and the other is a hybrid multistage test battery (MSTBH) design. The MSTB design consists of three tests: The first test used the AMI (approximate maximum information) method as the routing strategy; and as for the second and third, the “On-the-Fly” strategy (OMST) was employed. The MSTBH design also consists of three tests; …

Inferential Procedures For Dominance Analysis Measures In Multiple Regression, Shuwen Tang

Theses and Dissertations

In order to better interpret a selected multiple regression model, researchers are often interested in whether a predictor is significantly more important than another or not. This study investigates the performance of the Normal-Theory based (asymptotic) confidence interval and bootstrap confidence intervals for predictors' dominance relationships using both normal and non-normal data. The results show that asymptotic confidence interval method is adequate to make inferences for comparing two general dominance measures when the distribution is multivariate normal or slightly non-normal and when the effect size is no less than 0.15 and the sample size is at least 100. However, the …

Assessing Mathematical Competence In Second Language: Exploring Dif Evidences From Pisa Malaysian Data, Mazlina Husin

Theses and Dissertations

The year 2003 represents a significant milestone in the history of education development in Malaysia. From 2003, mathematics and science will be taught in English. This change in policy was deemed necessary to ensure that Malaysians are able to keep abreast with scientific and technological development that is mostly recorded in the English language. However, an unintended consequence of this language change was its huge impact on the national education system and the assessment of that system as well. Whenever students are not tested in their home language, one validity issue arises, which is how language, rather than the targeted …

Dif Analyses In Multilevel Data: Identification And Effects On Ability Estimates, Yao Wen

Theses and Dissertations

Fairness is an important issue in educational testing in that different groups of examinees should have equal probabilities of answering an item correctly, provided they have the same capabilities. Therefore, differential item functioning (DIF) analyses were developed due to the possibility of bias in cognitive or achievement tests. Data are multilevel structured in educational testing as students are nested within teachers who are nested within schools, and which may further be nested within districts. Although DIF analyses have been discussed for decades, they are rarely investigated in multilevel data. In this study, DIF analyses in multilevel data were investigated via …

Validity And Rater Reliability Of Peer And Self Assessments For Urban Middle School Students, Lucas Jackson

Theses and Dissertations

This project studied the validity and reliability of self and peer assessments used for group work. The targeted population is middle school students in urban schools. The sample includes 45 sixth graders selected from a public middle school in a large Midwestern metropolitan area. The students worked in groups to complete a classroom project. Self and peer assessment forms were used to rate each member's contribution to the group work. A Generalizability Theory design was used to evaluate the reliability of self and peer assessments. The validity of student ratings was assessed by comparing them to those assigned by the …

(Re)Envisioning Placement For 21st Century Writing Programs, Jessica Nastal-Dema

Theses and Dissertations

Writing assessment has continued to gain prominence within educational settings and public discourse in the United States throughout the past century. Placement into First-Year Writing, however, is consistently ignored in the scholarly literature, despite its central role within writing assessment and the university. This dissertation argues that placement is central to all students' university experiences, and deserves more attention. Placement is at the heart of composition: it affects each student, each instructor, each writing program, each institution. It significantly influences retention, instruction, budget, and even national reputation, since student retention and graduation rates are key factors in national rankings such …

Structures And Supports For Data Use In Schools: A Qualitative Case Study Of One Urban Elementary School, Anne Marie Groh

Theses and Dissertations

A federal policy in the United States has required sweeping changes in K-12 education. With No Child Left Behind legislation, schools are challenged to create conditions that bring each student to federally-set academic proficiency levels. Many schools have become more attentive to data about student performance and how it can inform their teaching decisions to meet federal goals under No Child Left Behind. How one urban elementary school in the Midwestern United States used data for decision-making in 2010-2011 was the focus of this research. The purpose of this study was to gain understanding about how personnel in one academically …

An Examination Of A Parochial Choice School Serving Latina/O Students: Perceptions, Realities And Promises, Tatiana Joseph

Theses and Dissertations

In Milwaukee, The Milwaukee Parental School Choice Program (MPCP) is a program that "allows low-income Milwaukee Students to attend participating private or religious schools located in Milwaukee with no charge for tuition if certain eligibility criteria are met" (Wisconsin Department of Public Instruction). The program allows for qualifying families to use a portion of the public educational funding selected (voucher) for their student to attend a public school or the private school of their choice. For low income families, school choice is an opportunity to flee from schools that struggle with dropouts, absenteeism and low achievement in hopes of a …