Statistical Methodology Commons^™

Assessment Of Method Effects Of Keying And Wording In Instruments: A Mixed-Methods Explanatory Sequential Study, Lin Ma

Electronic Theses and Dissertations

This dissertation presents an innovative approach to examining the keying method, wording method, and construct validity on psychometric instruments. By employing a mixed methods explanatory sequential design, the effects of keying and wording in two psychometric assessments were examined and validated. Those two self-report psychometric assessments were the Effortful Control assessment (Ellis & Rothbart, 2001) and the Grit assessment (Duckworth & Quinn, 2009). Moreover, the quantitative phase utilized structural equation modeling to analyze 2,104 students’ responses and assess the construct of keying and wording. Various hypothetical models were investigated and evaluated. The reliability of each construct in each method was …

Examining The Credibility Of Story-Based Causal Methodologies, Megan E. Kauffmann

Electronic Theses and Dissertations

The purpose of this study was to explore how evaluators justify using story-based methodologies when examining causality. The two primary research questions of the study included: 1) what arguments are made by evaluators to justify the credibility of story-based causal methodologies to evaluation stakeholders; and 2) from the perspective of evaluators, how do contextual factors influence whether story-based causal methodologies are perceived as credible by evaluation stakeholders? A case study was conducted to examine the cases of four evaluators who had experience implementing a story-based methodology in an evaluation. Data collection procedures included two interviews with each participant and a …

Mis-Specification Of Functional Forms In Growth Mixture Modeling: A Monte Carlo Simulation, Richa Ghevarghese

Electronic Theses and Dissertations

Growth mixture modeling (GMM) is a methodological tool used to represent heterogeneity in longitudinal datasets through the identification of unobserved subgroups following qualitatively and quantitatively distinct trajectories in a population. These growth trajectories or functional forms are informed by the underlying developmental theory, are distinct to each subgroup, and form the core assumptions of the model. Therefore, the accuracy of the assumed functional forms of growth strongly influences substantive research and theories of growth. While there is evidence of mis-specified functional forms of growth in GMM literature, the weight of this violation has been largely overlooked. Current solutions to circumvent …

Application Of An Organizational Evaluation Capacity Assessment In A Multinational Ngo: A Case Study To Support Applied Practice, Ryan James Smyth

Electronic Theses and Dissertations

As evaluation capacity building (ECB) has rapidly emerged as a practice in human service organizations and as a field of academic inquiry, attention has focused on methods of evaluation capacity building while assessment of organizational evaluation capacity (EC) has lagged behind. To examine the practice of organizational evaluation capacity assessment, this dissertation presents two separate but related studies. In sub-study 1, I present a qualitative evidence synthesis of the research theorizing organizational evaluation capacity models. In sub-study 2, I support the implementation of one of the tools from the evidence-synthesis at a multinational human service organization. I use a concurrent …

Evaluation Of The Effect Of The Clinical-Decision-Support Systems On Diabetes Management: A Multivariate Meta-Analysis Comparison With Univariate Meta-Analysis, Abdelfattah Elbarsha

Electronic Theses and Dissertations

The advantage of using meta-analysis lies in its ability in providing a quantitative summary of the findings from multiple studies. The aim of this dissertation was first to conduct a simulation study in order to understand what factors (sample size, between-study correlation, and percent of missing data) have a significant effect on meta-analysis estimates and whether using univariate or multivariate meta-analysis would produce different estimates.

The second goal of this study was to evaluate the effect of clinical decision support systems CDSS on diabetes care management by conducting three separate univariate meta-analyses and one multivariate meta-analysis. CDSS are health information …

The Combined Impact Of Continuous And Ordinal Auxiliary Variables On Missing Data Imputation In Sem, Salina Wu Whitaker

Electronic Theses and Dissertations

“Modern” methods of addressing missing data using full-information maximum-likelihood (FIML) have become mainstays in SEM analyses. FIML allows the inclusion of auxiliary variables which carry information that is related to missing values and can reduce bias in parameter estimates. Past research has illustrated the benefits of auxiliary variable inclusion under different missingness conditions (MCAR and MNAR; e.g., Enders, 2008), missingness proportions (e.g., Collins et al., 2001), and although limited, missingness patterns (e.g., Yoo, 2009) in FIML analyses. While past studies have focused on the effects of either continuous or ordinal auxiliary variables, no study has included both types in their …

Is The Reliability Of Objective Originality Scores Confounded By Elaboration?, Shannon Marie Maio

Electronic Theses and Dissertations

The increased use of text-mining models as a scoring mechanism for divergent thinking (DT) tasks has sparked concerns about the ways in which automated Originality scores may be influenced by other dimensions of DT, especially Elaboration. The debate centers around the question of whether too much variance in automated Originality scores is accounted for by the number of words a participant uses in a response (i.e., Elaboration), and, thus, how the influence of Elaboration can affect the reliability of Originality scores. Here, a partial correlation analysis, in conjunction with text-mining and psychometric modeling, is conducted to test the degree to …

A Comparison Of Bayesian Estimation Techniques In A Multidimensional Two-Parameter Partial Credit Item Response Model, Peiyan Liu

Electronic Theses and Dissertations

Bayesian estimation methods have shown better performance than the traditional Marginal Maximum Likelihood (MML) estimation method for parameter estimation in relatively simple item response models. However, extant literature is lacking on the investigation of Bayesian parameter estimation approaches for a multidimensional two parameter partial credit (M2PPC) model, therefore this simulation study investigated the performance of two Bayesian Markov Chain Monte Carlo (MCMC) algorithms: Gibbs Sampler and Hamiltonian Monte Carlo-No-U-Turn-Sampler (HMC-NUTS) for M2PPC models' parameter estimation. It compared the estimation accuracy and computing speed in different combinations of situations, including prior choices, test lengths, and the relationships between dimensions.

The datasets …