Open Access. Powered by Scholars. Published by Universities.®
- Keyword
- File Type
Articles 1 - 7 of 7
Full-Text Articles in Quantitative Psychology
Many-Facet Rasch Designs: How Should Raters Be Assigned To Examinees?, Christine E. Demars, Yelisey A. Shapovalov, John D. Hathcoat
Many-Facet Rasch Designs: How Should Raters Be Assigned To Examinees?, Christine E. Demars, Yelisey A. Shapovalov, John D. Hathcoat
Department of Graduate Psychology - Faculty Scholarship
In Facets models, raters should be connected, and there are multiple ways to connect raters. Keeping the number of ratings constant and two raters scoring each examinee, the standard error of both rater severity and examinee ability was higher when raters scored one examinee in common with many different raters than when they scored many examinees in common with two raters. However, the differences were small, especially for the standard error of examinee ability. Alternatively, when only a subset of examinees were scored by two or more raters, the smallest standard errors were achieved when all raters scored a common …
Differential Motivation In Remote Educational Assessment: Person-Based Filtering Versus Response-Based Filtering, Sarah Alahmadi, Christine E. Demars
Differential Motivation In Remote Educational Assessment: Person-Based Filtering Versus Response-Based Filtering, Sarah Alahmadi, Christine E. Demars
Department of Graduate Psychology - Faculty Scholarship
Large-scale educational assessments are often considered low-stakes, increasing the possibility of confounding true performance level with low motivation. These concerns are amplified in remote testing conditions. To remove the effects of low effort levels in responses observed in remote low-stakes testing, several motivation filtering methods can be used to purify the data. We estimated scores from assessment data collected remotely in Spring 2021 six ways, applying examinee-based filtering methods (filtering examinees based on total time) and response-based filtering methods (filtering responses using the effort-moderated IRT model), varying the thresholds selected to separate solution behavior (SB) responses from rapid-guessing behavior (RGB). …
Item Parameter Recovery With And Without The Use Of Priors, Paulius Satkus, Christine E. Demars
Item Parameter Recovery With And Without The Use Of Priors, Paulius Satkus, Christine E. Demars
Department of Graduate Psychology - Faculty Scholarship
Marginal maximum likelihood (MML), a common estimation method for IRT models, is not inherently a Bayesian procedure. However, due to estimation difficulties, Bayesian priors are often applied to the likelihood when estimating 3PL models, especially with small samples. Little focus has been placed on choosing the priors for MML estimation. In this study, using samples sizes of 1000 or smaller, not using priors often led to extreme, implausible parameter estimates. Applying prior distributions to the c-parameters alleviated the estimation problems with samples of 1000; priors on both the a-parameters and c-parameters were needed for the samples of …
Examining The Performance Of The Alignment Method In Dif Analyses, Paulius Satkus, Christine E. Demars
Examining The Performance Of The Alignment Method In Dif Analyses, Paulius Satkus, Christine E. Demars
Department of Graduate Psychology - Faculty Scholarship
The alignment procedure is a new method for multiple group invariance models. An important advantage of alignment over the traditional methods is that alignment does not require full measurement invariance to estimate group means and variances (Muthén & Asparouhov, 2014). Simulation studies have supported that alignment performs adequately in situations when few items are noninvariant (or function differentially across groups – DIF). In most other studies, the tests were simulated to represent attitudinal surveys (e.g., fewer items, continuous data). In this study, we evaluated how alignment would perform with a typical educational cognitive test – 40 items scored dichotomously. Different …
Examining The Effects Of Specifying Bayesian Priors On The Wald's Test For Dif, Paulius Satkus, Christine E. Demars
Examining The Effects Of Specifying Bayesian Priors On The Wald's Test For Dif, Paulius Satkus, Christine E. Demars
Department of Graduate Psychology - Faculty Scholarship
No abstract provided.
An Applied Example Of A Two-Tier Multiple-Group Testlet Model, Paulius Satkus, Christine E. Demars
An Applied Example Of A Two-Tier Multiple-Group Testlet Model, Paulius Satkus, Christine E. Demars
Department of Graduate Psychology - Faculty Scholarship
No abstract provided.
Considerations In S-Χ2: Rest Score Or Summed Score, Priors, And Violations Of Normality, Christine E. Demars, Derek Sauder
Considerations In S-Χ2: Rest Score Or Summed Score, Priors, And Violations Of Normality, Christine E. Demars, Derek Sauder
Department of Graduate Psychology - Faculty Scholarship
The S-χ2 item fit index is one of the few item fit indices that appears to maintain accurate Type I error rates. This study explored grouping examinees by the rest score or summed score, prior distributions for the item parameters, and the shape of the ability distribution. Type I error was slightly closer to the nominal level for the total-score S-χ2 for the longest tests, but power was higher for the rest-score S-χ2 in every condition where power was < 1. Prior distributions reduced the proportion of estimates with extreme standard errors but slightly inflated the Type I error rates in some conditions. When the ability distribution was not normally distributed, integrating over an empirically-estimated distribution yielded Type I error rates closer to the nominal value than integrating over a normal distribution.