Quantitative Psychology Commons^™

Applying Conditional Distributions To Individuals: Using Latent Variable Models, Feng Ji

Theses and Dissertations

This study proposes a new method to interpret individual results of psychological test batteries. The Mahalanobis distance is a commonly-used measure of how unusual an individual’s profile of scores is compared to a population of score profiles. In models in which there is a set of predictors and a set of dependent variables (e.g., cognitive abilities predicting academic abilities), it is useful to distinguish between a profile of dependent scores that is unusual because its profile of predictor scores is unusual and a profile of dependent scores that is unusual even after controlling for the predictors. The conditional Mahalanobis distance …

Posterior Predictive Model Checking Of Local Misfit For Bayesian Confirmatory Factor Analysis, Chi Hang Au

Masters Theses, 2010-2019

Posterior predictive model checks (PPMC) are one Bayesian model-data fit approach. Thus far, PPMC for Confirmatory Factor Analytic applications focused primarily on global fit evaluation, ignoring the nuanced information in local misfit diagnostics. This study developed a PPMC approach for local misfit and applied it to a test-taking motivation scale. If the PPMC approach is effective, fit conclusions derived from the PPMC approach should be congruent with the fit conclusions derived from the Frequentist approach. Number of item-pairs flagged as misfitting and number of disagreements were computed to evaluate congruence. Congruence is achieved if the number of item-pairs flagged as …

The Trouble With Test Banks, Harvey Richman, Molly Hrezo

Perspectives In Learning

We compared the psychometrics of quiz questions randomly selected from a test bank with the psychometrics of quiz questions the instructor had selected from the bank for quality and modified (if necessary). On multiple psychometric indices, the instructor selected/modified questions were superior to questions randomly selected from the test bank. Most notably, when compared with instructor written/modified questions, randomly selected bank questions were nearly 6.5 times more likely to contain a distractor that drew more responses than the correct answer. Details and implications are discussed.

Retrospective Versus Prospective Measurement Of Examinee Motivation In Low-Stakes Testing Contexts: A Moderated Mediation Model, Aaron J. Myers

Masters Theses, 2010-2019

Expectancy-value theory applied to examinee motivation suggests examinees’ perceived value of a test indirectly affects test performance via examinee effort. This empirically supported indirect effect, however, is often modeled using importance and effort scores measured after test completion, which does not align with their theoretically specified temporal order. Retrospectively measured importance and effort scores may be influenced by examinees’ test performance, impacting the estimate of the indirect effect. To investigate the effect of timing of measurement, first-year college students were randomly assigned to one of three conditions where (1) importance and effort were measured retrospectively; (2) importance was measured prospectively; …

Student Learning Gains In Higher Education: A Longitudinal Analysis With Faculty Discussion, Catherine E. Mathers

Masters Theses, 2010-2019

Student learning is the primary desired outcome of a college education. To understand how educational programming and curricula affect students, colleges and universities must collect evidence of student learning gain. In this study, a longitudinal design was employed to investigate how a math and science general education curriculum impacted college students’ quantitative and scientific reasoning. Quantitative and scientific reasoning gain scores were computed and predicted from personal (i.e., prior knowledge, gender) and curriculum (i.e., number of completed courses in the domain) characteristics to uncover what factors relate to learning gain. Collapsing across personal and curriculum variables, gain scores were moderate …

You Only Live Up To The Standards You Set: An Evaluation Of Different Approaches To Standard Setting, Scott N. Strickman

Dissertations, 2014-2019

Interpretation of performance in reference to a standard can provide nuanced, finely-tuned information regarding examinee abilities beyond that of just a total score. However, there is a multitude of ways to set performance standards yet little guidance regarding which method operates best and under what circumstances. Traditional methods are the most common approach adopted in practice and heavily involve subject matter experts (SMEs). Two other approaches have been suggested in the literature as alternative ways to set performance standards, although they have yet to be implemented in practice. Data-driven approaches do not involve SMEs but rather rely solely upon statistical …

Strategies And Resources To Enhance Test Evaluation And Selection, Janet F. Carlson, Nancy Anderson

Buros Center: Professional Staff Publications

Testing serves an important function for SLPs in offering an evidence base that is useful in screening, diagnosing, monitoring progress, and documenting outcomes. Tests are used to measure diverse constructs such as communication, literacy, oral and written language, receptive and expressive vocabulary, articulation, phonological awareness and processing, and auditory perception and processing. In addition, specific impairments may require specialized measures to evaluate conditions such as stuttering and orthographic competence.

When using tests to diagnose language impairments, Betz, Eickhoff, and Sullivan (2013) suggest that SLPs consider carefully a test’s psychometric properties, particularly because of the “increasing emphasis on evidence-based practice, specifically, …

The Effects Of A Planned Missingness Design On Examinee Motivation And Psychometric Quality, Matthew S. Swain

Dissertations, 2014-2019

Assessment practitioners in higher education face increasing demands to collect assessment and accountability data to make important inferences about student learning and institutional quality. The validity of these high-stakes decisions is jeopardized, particularly in low-stakes testing contexts, when examinees do not expend sufficient motivation to perform well on the test. This study introduced planned missingness as a potential solution. In planned missingness designs, data on all items are collected but each examinee only completes a subset of items, thus increasing data collection efficiency, reducing examinee burden, and potentially increasing data quality. The current scientific reasoning test served as the Long …

Examining The Performance Of The Metropolis-Hastings Robbins-Monro Algorithm In The Estimation Of Multilevel Multidimensional Irt Models, Bozhidar M. Bashkov

Dissertations, 2014-2019

The purpose of this study was to review the challenges that exist in the estimation of complex (multidimensional) models applied to complex (multilevel) data and to examine the performance of the recently developed Metropolis-Hastings Robbins-Monro (MH-RM) algorithm (Cai, 2010a, 2010b), designed to overcome these challenges and implemented in both commercial and open-source software programs. Unlike other methods, which either rely on high-dimensional numerical integration or approximation of the entire multidimensional response surface, MH-RM makes use of Fisher’s Identity to employ stochastic imputation (i.e., data augmentation) via the Metropolis-Hastings sampler and then apply the stochastic approximation method of Robbins and Monro …

Validation Of The Item-Attribute Matrix In Timss-Mathematics Using Multiple Regression And The Lsdm, Lin Ma

Electronic Theses and Dissertations

For many cognitive diagnostic models, the item-attribute matrix (or Q-matrix) is an essential component which displays the relationship between items and their latent attributes or skills in knowledge and cognitive processes. However, it is a challenge to develop an effective Q-matrix.The purposes of this study were (1) to validate of the item-attribute matrix using two levels of attributes (Level 1 attributes and Level 2 sub-attributes), and (2) through retrofitting the diagnostic models to the mathematics test of the Trends in International Mathematics and Science Study (TIMSS), to evaluate the construct validity of TIMSS mathematics assessment by comparing the results of …

The Reliability And Validity Of The Thin Slice Technique: Observational Research On Video Recorded Medical Interactions, Tanina Suzanne Foster

Wayne State University Dissertations

The Reliability and Validity of the Thin Slice Technique: Observational Research on Video Recorded Medical Interactions

Introduction: Observational research using the thin slice technique has been routinely incorporated in observational research methods, however there is limited evidence supporting use of this technique compared to full interaction coding. The purpose of this study was to determine if this technique could be reliability coded, if ratings are consistent between the first, second and third slice, and if they are indeed representative of full interactions.

Methods: Three 30-second thin slices were sampled from the beginning, middle and end of a full-length video-recorded …

Teacher Support Mediates Concurrent And Longitudinal Associations Between Temperament And Mild Depressive Symptoms In Sixth Grade, Kathleen Moritz Rudasill, Patrick Pössel, Stephanie Winkeljohn Black, Kate Niehaus

Department of Educational Psychology: Faculty Publications

The combination of changes occurring at the transition to middle school may be a catalyst for the onset of depressive symptoms, yet teacher support at this transition is protective. Research points to certain temperamental traits as risk factors for developing depressive symptoms. This study examines student reports of teacher support and teacher reports of student–teacher relationship (STR) quality as mediators of associations between child temperament (i.e. negative emotionality at age 4½ : and emotional reactivity in elementary grades) and depressive symptoms in sixth grade. Results indicate (a) negative emotionality predicted emotional reactivity and depressive symptoms; (b) emotional reactivity predicted depressive …

An Analysis Of Factor Extraction Strategies: A Comparison Of The Relative Strengths Of Principal Axis, Ordinary Least Squares, And Maximum Likelihood In Research Contexts That Include Both Categorical And Continuous Variables, Kevin Barry Coughlin

USF Tampa Graduate Theses and Dissertations

This study is intended to provide researchers with empirically derived guidelines for conducting factor analytic studies in research contexts that include dichotomous and continuous levels of measurement. This study is based on the hypotheses that ordinary least squares (OLS) factor analysis will yield more accurate parameter estimates than maximum likelihood (ML) and principal axis factor anlaysis (PAF); the level of improvement in estimates will be related to the proportion of observed variables that are dichotomized and the strength of communalities within the data sets.

To achieve this study's objective, maximum likelihood, ordinary least squares, and principal axis factor extraction models …

A Preliminary Investigation Of The Validity Of Time-Based Measures Of Sustained Attention For Children, Michael R. Kulfan

Antioch University Dissertations & Theses

This study is a preliminary investigation of the validity of using time-based measures to quantify sustained attention in children ages 6-12. Problems with sustained attention negatively affect childhood learning and development. The prevalence of disorders known to impact sustained attention performance continue to rise in the United States. Currently, commercially available, objective measures of sustained attention use normative comparisons that provide limited information about the effect such problems have on child performance in natural settings. We reviewed test data from 290 charts of children ages 6-12 referred for neuropsychological evaluation. The Test of Everyday Attention for Children (TEA-Ch) is an …

The Dependent Samples T And Wilcoxon Sign Rank Maximum Test, Saverpierre Maggio

Wayne State University Dissertations

A maximum test using the parametric dependent samples t-test and the non-parametric Wilcoxon sign rank test was created using a FORTRAN program and various subroutines of the International Mathematical and Statistical Libraries (IMSL, 1980). Two tailed critical values were derived from a mixed normal distribution. Critical values obtained were at the 0.05, 0.025, 0.01 and 0.005 alpha levels via sample sizes (n) 8 through 30, 45, 60, 90 and 120. Critical values were compared to values obtained through the application of the Bonferroni correction method. It was concluded that the Bonferroni is an unnecessary method. Findings of the study are …

Improving Irt Parameter Estimates With Small Sample Sizes: Evaluating The Efficacy Of A New Data Augmentation Technique, Brett P. Foley

College of Education and Human Sciences: Dissertations, Theses, and Student Research

The 3PL model is a flexible and widely used tool in assessment. However, it suffers from limitations due to its need for large sample sizes. This study introduces and evaluates the efficacy of a new sample size augmentation technique called Duplicate, Erase, and Replace (DupER) Augmentation through a simulation study. Data are augmented using several variations of DupER Augmentation (based on different imputation methodologies, deletion rates, and duplication rates), analyzed in BILOG-MG 3, and results are compared to those obtained from analyzing the raw data. Additional manipulated variables include test length and sample size. Estimates are compared using seven different …