Open Access. Powered by Scholars. Published by Universities.®
- Keyword
-
- DIF (2)
- Item response theory (2)
- Motivation (2)
- 3PL (1)
- AAVE (1)
-
- Ability distribution (1)
- Bayesian inference (1)
- Bifactor (1)
- Confirmatory factor analysis (1)
- Data-driven approaches (1)
- Estimation methods (1)
- Examinee effort (1)
- Examinee motivation (1)
- Faculty expectations (1)
- Heatmap (1)
- Higher education (1)
- Higher education assessment (1)
- IRT (1)
- IRTree models (1)
- Integrated approaches (1)
- Item fit (1)
- Latent Class Analysis (1)
- Learning gains (1)
- Learning improvement (1)
- Longitudinal (1)
- Longitudinal methods (1)
- Lord's chi-square (1)
- MIRT (1)
- Measurement (1)
- Mediation (1)
- Publication
- Publication Type
Articles 1 - 13 of 13
Full-Text Articles in Quantitative Psychology
Writing While Black: African American Vernacular English (Aave) And Perceived Writing Performance, Jaylin N. Nesbitt
Writing While Black: African American Vernacular English (Aave) And Perceived Writing Performance, Jaylin N. Nesbitt
Masters Theses, 2020-current
In the education system, there have historically been inequities that have severely disadvantaged Black students academically. One area in which these inequities surface is on writing assessments in the form of lower scores. I argue that because the U.S. education system is centered around Standard American English (SAE), it disadvantages those from different linguistic backgrounds, specifically Black students, as they are most likely to be speakers of African American Vernacular English (AAVE). Although there are theoretical justifications for this, past literature has not empirically tied inequities on writing assessments to Black students’ use of AAVE. The current study used Natural …
Getting Caught-Up In The Process: Does It Really Matter?, Nikole Gregg
Getting Caught-Up In The Process: Does It Really Matter?, Nikole Gregg
Dissertations, 2020-current
Likert items are the most commonly used item-type for measuring attitudes and beliefs. However, responses from Likert items are often plagued with construct-irrelevant variance due to response style behavior. In other words, variability from Likert-item scores can be parsed into: 1) variance pertinent to the construct or trait of interest, and 2) variance irrelevant to the construct or trait of interest. Multidimensional Item Response Theory (MIRT) is an increasingly common modeling approach to parse out information regarding the response style traits and the trait of interest. These MIRT approaches are categorized into threshold-based approaches and response process approaches. An increasingly …
Examining The Effects Of Specifying Bayesian Priors On The Wald's Test For Dif, Paulius Satkus, Christine E. Demars
Examining The Effects Of Specifying Bayesian Priors On The Wald's Test For Dif, Paulius Satkus, Christine E. Demars
Department of Graduate Psychology - Faculty Scholarship
No abstract provided.
An Applied Example Of A Two-Tier Multiple-Group Testlet Model, Paulius Satkus, Christine E. Demars
An Applied Example Of A Two-Tier Multiple-Group Testlet Model, Paulius Satkus, Christine E. Demars
Department of Graduate Psychology - Faculty Scholarship
No abstract provided.
Test Emotions, Value, And Self-Efficacy: A Longitudinal Model Predicting Examinee Effort And Performance On A Low-Stakes Test, Paulius Satkus
Test Emotions, Value, And Self-Efficacy: A Longitudinal Model Predicting Examinee Effort And Performance On A Low-Stakes Test, Paulius Satkus
Masters Theses, 2010-2019
The validity of scores from low-stakes tests may be compromised by examinee motivation. Expectancy-Value theory (EV) has been used to frame the antecedents of examinee motivation in low-stakes testing contexts. According to EV theory, the perceived value of the test and the expectancy to succeed on the test directly affect examinee effort, which then affects test performance. Cross-sectional research studies in low-stakes testing contexts offer some support of EV theory. Control-Value theory (CV) serves as another theory to understand motivation toward a task. CV theory encompasses the constructs of expectancy and value from EV theory, but incorporates test emotions as …
Considerations In S-Χ2: Rest Score Or Summed Score, Priors, And Violations Of Normality, Christine E. Demars, Derek Sauder
Considerations In S-Χ2: Rest Score Or Summed Score, Priors, And Violations Of Normality, Christine E. Demars, Derek Sauder
Department of Graduate Psychology - Faculty Scholarship
The S-χ2 item fit index is one of the few item fit indices that appears to maintain accurate Type I error rates. This study explored grouping examinees by the rest score or summed score, prior distributions for the item parameters, and the shape of the ability distribution. Type I error was slightly closer to the nominal level for the total-score S-χ2 for the longest tests, but power was higher for the rest-score S-χ2 in every condition where power was < 1. Prior distributions reduced the proportion of estimates with extreme standard errors but slightly inflated the Type I error rates in some conditions. When the ability distribution was not normally distributed, integrating over an empirically-estimated distribution yielded Type I error rates closer to the nominal value than integrating over a normal distribution.
Posterior Predictive Model Checking Of Local Misfit For Bayesian Confirmatory Factor Analysis, Chi Hang Au
Posterior Predictive Model Checking Of Local Misfit For Bayesian Confirmatory Factor Analysis, Chi Hang Au
Masters Theses, 2010-2019
Posterior predictive model checks (PPMC) are one Bayesian model-data fit approach. Thus far, PPMC for Confirmatory Factor Analytic applications focused primarily on global fit evaluation, ignoring the nuanced information in local misfit diagnostics. This study developed a PPMC approach for local misfit and applied it to a test-taking motivation scale. If the PPMC approach is effective, fit conclusions derived from the PPMC approach should be congruent with the fit conclusions derived from the Frequentist approach. Number of item-pairs flagged as misfitting and number of disagreements were computed to evaluate congruence. Congruence is achieved if the number of item-pairs flagged as …
You Only Live Up To The Standards You Set: An Evaluation Of Different Approaches To Standard Setting, Scott N. Strickman
You Only Live Up To The Standards You Set: An Evaluation Of Different Approaches To Standard Setting, Scott N. Strickman
Dissertations, 2014-2019
Interpretation of performance in reference to a standard can provide nuanced, finely-tuned information regarding examinee abilities beyond that of just a total score. However, there is a multitude of ways to set performance standards yet little guidance regarding which method operates best and under what circumstances. Traditional methods are the most common approach adopted in practice and heavily involve subject matter experts (SMEs). Two other approaches have been suggested in the literature as alternative ways to set performance standards, although they have yet to be implemented in practice. Data-driven approaches do not involve SMEs but rather rely solely upon statistical …
Retrospective Versus Prospective Measurement Of Examinee Motivation In Low-Stakes Testing Contexts: A Moderated Mediation Model, Aaron J. Myers
Retrospective Versus Prospective Measurement Of Examinee Motivation In Low-Stakes Testing Contexts: A Moderated Mediation Model, Aaron J. Myers
Masters Theses, 2010-2019
Expectancy-value theory applied to examinee motivation suggests examinees’ perceived value of a test indirectly affects test performance via examinee effort. This empirically supported indirect effect, however, is often modeled using importance and effort scores measured after test completion, which does not align with their theoretically specified temporal order. Retrospectively measured importance and effort scores may be influenced by examinees’ test performance, impacting the estimate of the indirect effect. To investigate the effect of timing of measurement, first-year college students were randomly assigned to one of three conditions where (1) importance and effort were measured retrospectively; (2) importance was measured prospectively; …
Student Learning Gains In Higher Education: A Longitudinal Analysis With Faculty Discussion, Catherine E. Mathers
Student Learning Gains In Higher Education: A Longitudinal Analysis With Faculty Discussion, Catherine E. Mathers
Masters Theses, 2010-2019
Student learning is the primary desired outcome of a college education. To understand how educational programming and curricula affect students, colleges and universities must collect evidence of student learning gain. In this study, a longitudinal design was employed to investigate how a math and science general education curriculum impacted college students’ quantitative and scientific reasoning. Quantitative and scientific reasoning gain scores were computed and predicted from personal (i.e., prior knowledge, gender) and curriculum (i.e., number of completed courses in the domain) characteristics to uncover what factors relate to learning gain. Collapsing across personal and curriculum variables, gain scores were moderate …
The Effects Of A Planned Missingness Design On Examinee Motivation And Psychometric Quality, Matthew S. Swain
The Effects Of A Planned Missingness Design On Examinee Motivation And Psychometric Quality, Matthew S. Swain
Dissertations, 2014-2019
Assessment practitioners in higher education face increasing demands to collect assessment and accountability data to make important inferences about student learning and institutional quality. The validity of these high-stakes decisions is jeopardized, particularly in low-stakes testing contexts, when examinees do not expend sufficient motivation to perform well on the test. This study introduced planned missingness as a potential solution. In planned missingness designs, data on all items are collected but each examinee only completes a subset of items, thus increasing data collection efficiency, reducing examinee burden, and potentially increasing data quality. The current scientific reasoning test served as the Long …
Extending An Irt Mixture Model To Detect Random Responders On Non-Cognitive Polytomously Scored Assessments, Mandalyn R. Swanson
Extending An Irt Mixture Model To Detect Random Responders On Non-Cognitive Polytomously Scored Assessments, Mandalyn R. Swanson
Dissertations, 2014-2019
This study represents an attempt to distinguish two classes of examinees – random responders and valid responders – on non-cognitive assessments in low-stakes testing. The majority of existing literature regarding the detection of random responders in low-stakes settings exists in regard to cognitive tests that are dichotomously scored. However, evidence suggests that random responding occurs on non-cognitive assessments, and as with cognitive measures, the data derived from such measures are used to inform practice. Thus, a threat to test score validity exists if examinees’ response selections do not accurately reflect their underlying level on the construct being assessed. As with …
Examining The Performance Of The Metropolis-Hastings Robbins-Monro Algorithm In The Estimation Of Multilevel Multidimensional Irt Models, Bozhidar M. Bashkov
Examining The Performance Of The Metropolis-Hastings Robbins-Monro Algorithm In The Estimation Of Multilevel Multidimensional Irt Models, Bozhidar M. Bashkov
Dissertations, 2014-2019
The purpose of this study was to review the challenges that exist in the estimation of complex (multidimensional) models applied to complex (multilevel) data and to examine the performance of the recently developed Metropolis-Hastings Robbins-Monro (MH-RM) algorithm (Cai, 2010a, 2010b), designed to overcome these challenges and implemented in both commercial and open-source software programs. Unlike other methods, which either rely on high-dimensional numerical integration or approximation of the entire multidimensional response surface, MH-RM makes use of Fisher’s Identity to employ stochastic imputation (i.e., data augmentation) via the Metropolis-Hastings sampler and then apply the stochastic approximation method of Robbins and Monro …