Education Commons^™

Missing Data In Complex Sample Surveys: Impact Of Deletion And Imputation Treatments On Point And Interval Parameter Estimates, Anh Pham Kellermann

USF Tampa Graduate Theses and Dissertations

The purpose of this simulation study was to evaluate the relative performance of five missing data treatments (MDTs) for handling missing data in complex sample surveys. The five missing data methods included in this study were listwise deletion (LW), single hot-deck imputation (HS), single regression imputation (RS), hot-deck-based multiple imputation (HM), and regression-based multiple imputation (RM). These MDTs were assessed in the context of regression weight estimates in multiple regression analysis in complex sample data with two data levels. In this study, the multiple regression equation had six regressors without missing data and two regressors with missing data. The four …

Investigating Students’ Basic Needs And Motivation In College Chemistry Courses With The Lens Of Self-Determination Theory, Yujuan Liu

USF Tampa Graduate Theses and Dissertations

More graduates in science, technology, engineering, and mathmetics (STEM) fields are needed to keep our nation’s preeminance in the global fields of technology and science. However, fewer than 40% of students who intend to major in STEM fields when entering college complete a STEM degree. Therefore, it is important to explore factors to improve student persistence in STEM fields at the college level as well as to understand the interrelationship between student motivation, academic achievement, and persistence. Motivation is strongly associated with student achievement and persistence; meanwhile, academic achievement can also affect persistence. Self-determination theory (SDT) represents a framework of …

The Empirical Selection Of Anchor Items Using A Multistage Approach, Brandon Craig

USF Tampa Graduate Theses and Dissertations

The purpose of this study was to determine if using a multistage approach for the empirical selection of anchor items would lead to more accurate DIF detection rates than the anchor selection methods proposed by Kopf, Zeileis, & Strobl (2015b). A simulation study was conducted in which the sample size, percentage of DIF, and balance of DIF were manipulated. The outcomes of interest were true positive rates, false positive rates, familywise false positive rates, anchor contamination rates, and familywise anchor contamination rates. Results showed the proposed multistage methods produced lower anchor contamination rates than the non-multistage methods under some conditions, …

Learner-Directed Vs. Instructor-Provided Curriculum Among Undergraduate Students, Christopher D. Martinez

USF Tampa Graduate Theses and Dissertations

There has been little or no research on the use of a Learning Object in the field of religious studies. The purpose of this research study was to assess undergraduate student knowledge gain outcomes between instructor-provided and learner-directed methods, which is the independent variable, using asynchronous, online, narrated PowerPoint presentations compared to undergraduate student knowledge gain using an online learning-objects unit, in a college-level world religions survey course at St. Petersburg College. This study assessed undergraduate student knowledge gain outcomes between instructor-provided and learner-directed methods among 90 college undergraduate students and determined whether there was a significant difference in the …

Perceptual Learning Style Modalities: Comparing Latino, Black, And Caucasian Adults, Nicolle Chantelle Hardy

USF Tampa Graduate Theses and Dissertations

Abstract

The purpose of this study was to compare the individual learning modalities of Latino, Black, and Caucasian males and females with at least some college education utilizing the Multi-modal Paired Associates Learning Test IV (MMPALT IV). Using the MMPALT IV, 20 participants from each of the three race/ethnicities above the age of 40 were measured in each of the seven perceptual modalities: Visual, Print, Aural, Interactive, Haptic, Kinesthetic, and Olfactory. The MMPALT IV is a performance-based test, which measures a person’s capacity to acquire information through each of the seven learning channels.

ANOVA tests (2 x 3) with a …

Extending The Model With Internal Restrictions On Item Difficulty (Mirid) To Study Differential Item Functioning, Yong "Isaac" Li

USF Tampa Graduate Theses and Dissertations

Differential item functioning (DIF) is a psychometric issue routinely considered in educational and psychological assessment. However, it has not been studied in the context of a recently developed componential statistical model, the model with internal restrictions on item difficulty (MIRID; Butter, De Boeck, & Verhelst, 1998). Because the MIRID requires test questions measuring either single or multiple cognitive processes, it creates a complex environment for which traditional DIF methods may be inappropriate. This dissertation sought to extend the MIRID framework to detect DIF at the item-group level and the individual-item level. Such a model-based approach can increase the interpretability of …

Many Hands Make Light Work: Crowdsourced Ratings Of Medical Student Osce Performance, Mark Grichanik

USF Tampa Graduate Theses and Dissertations

Clinical skills are often measured using objective structured clinical examinations (OSCEs) in healthcare professions education programs. As with assessment centers, it is challenging to provide learners with effective feedback due to burdensome human capital demands. The aim of this dissertation was to evaluate the viability of using a crowdsourced system to gather OSCE ratings and feedback. Aggregating evaluations of student performance from a crowd of patient proxies has the potential to mitigate biases associated with single-rater evaluations, allow the patient a voice as the consumer of physician behavior, improve reliability, reduce costs, improve feedback latency, and help learners develop a …

Exploring Teacher Assessment Literacy Through The Process Of Training Teachers To Write Assessment Items, Heather Peltier Wright

USF Tampa Graduate Theses and Dissertations

The purpose of this study was to examine the process and impact of assessment training content and delivery mode on the quality of assessment items developed by the teachers in a two-year assessment development project. Teacher characteristics were examined as potential moderating factors. Four types of delivery mode were employed in the project: synchronous online, asynchronous online, in-person workshop, and blended (a combination of online and in-person training). The quality of assessment items developed by participating teachers was measured via: 1) item acceptance rate, 2) number of item reviews (as an indicator of how many times accepted items were rejected …

Exploring The Test Of Covariate Moderation Effect And The Impact Of Model Misspecification In Multilevel Mimic Models, Chunhua Cao

USF Tampa Graduate Theses and Dissertations

In multilevel MIMIC models, covariates at the between level and at the within level can be modeled simultaneously. Covariates interaction effect occurs when the effect of one covariate on the latent factor varies depending on the level of the other covariate. The two covariates can be both at the between level, both at the within level, and one at the between level and the other one at the within level. And they can create between level covariates interaction, within level covariates interaction, and cross level covariates interaction. Study One purports to examine the performance of multilevel MIMIC models in estimating …

Examining Content Validity Of The Nurse Competence Scale In The United States, Jessica B. Shearer

USF Tampa Graduate Theses and Dissertations

Nursing competence is a necessity as it affects public health, safety and welfare; however there are very few reliable and valid tools that measure nursing competence. The Nurse Competence Scale has been used in many different countries outside of the United States and is based on Benner’s Novice to Expert theory. The purpose of this study was to examine the content validity of the Nurse Competence Scale using a mixed methods approach. Descriptive statistics were used to report results of the frequency of use scale and the critical to practice scale. Nursing faculty indicate Prioritizing My Activities Flexibility According to …

Examinees' Perceptions Of The Physical Aspects Of The Testing Environment During The National Physical Therapy Examination, Ellen Kroog Donald

USF Tampa Graduate Theses and Dissertations

Despite the increasing number of individuals taking computer-based tests, little is known about how examinees perceive computer-based testing environments and the extent to which these testing environments are perceived to affect test performance. The purpose of the present study was to assess the testing environment as perceived by individuals taking the National Physical Therapy Examination (NPTE), a high-stakes licensure examination. Perceptions of the testing environments were assessed using an examinee self-report questionnaire. The questionnaire included items that measured individuals’ preference and perception of specific characteristics of the environment, along with demographic information and one open-ended item. Questionnaires were distributed by …

Multilevel Analysis Of A Scale Measuring Educators’ Perceptions Of Multi-Tiered Systems Of Supports Practices, Leslie Marie Marshall

USF Tampa Graduate Theses and Dissertations

This study aimed to provide evidence of reliability and validity for the 42-item Perceptions of Practices Survey. The scale was designed to assess educators’ perceptions of the extent to which their schools were implementing multi-tiered system of supports (MTSS) practices. The survey was initially given as part of a larger evaluation project of a 3-year, statewide initiative designed to evaluate MTSS implementation. Elementary educators (Level-1 n = 2,109, Level-2 n = 62) completed the survey in September/October of 2007, September/October of 2008 (Level-1 n = 1,940, Level-2 n = 61), and January/February of 2010 (Level-1 n = 2,058, Level-2 …

Motivation For Mathematics: The Development And Initial Validation Of An Abbreviated Instrument, Kenneth Lee Butler

USF Tampa Graduate Theses and Dissertations

This study outlines the development and initial validation of an abbreviated instrument intended to measure motivation for mathematics of university students in developmental algebra courses. I look across many of the predominant theories on motivation with the aim of representing several of these theories as latent constructs in a single instrument that is short enough to be administered in a reasonable amount of time, but inclusive enough that it could incorporate subscales representing multiple distinct latent factors. This study answers a call by researchers expressing a need to investigate relationships between disparate theories on motivation and is a response to …

A Comparison Of Educational "Value-Added" Methodologies For Classifying Teacher Effectiveness: Value Tables Vs. Covariate Regression, Theodore J. Dwyer

USF Tampa Graduate Theses and Dissertations

There is a great deal of concern regarding teacher impacts on student achievement being used as a substantial portion of a teacher’s performance evaluation. This study investigated the degree of concordance and discordance between mathematics teacher ranking using value tables and covariate regression, which have both been used as measures for teacher effectiveness. The researcher examined teacher rankings, before and after the state recommended classification, using correlational techniques, comparison matrices, and visual examination for value-added scores derived from the value table versus the covariate regression approach. Examination demonstrated strong correlations between the initial rankings (r = .77 to .98) …

An Empirical Comparison Of The Effect Of Missing Data On Type I Error And Statistical Power Of The Likelihood Ratio Test For Differential Item Functioning: An Item Response Theory Approach Using The Graded Response Model, Patricia Rodriguez De Gil

USF Tampa Graduate Theses and Dissertations

In the context of educational research, missing data arise when examinees omit or do not reach an item, which generates an item nonresponse problem. Using a simulation approach, in addition to conducting complete data analyses, this study compared the performance of six methods for treating item nonresponse in the context of differential item functioning (DIF). The effect of missing data on the Type I error and statistical power of the Likelihood Ratio test for DIF detection in small scales was examined in the context of Item Response Theory (IRT-LR), using polytomous, Likert-type data and the graded response model. The effect …

What You Know Counts: Why We Should Elicit Prior Probabilities From Experts To Improve Quantitative Analysis With Qualitative Knowledge In Special Education Science, Tyler Aaron Hicks

USF Tampa Graduate Theses and Dissertations

Qualitative knowledge is about types of things, and their excellences. There are many ways we humans produce qualitative knowledge about the world, and much of it is derived from non-quantitative sources (e.g., narratives, clinical experiences, intuitions). The purpose of my dissertation was to investigate the possibility of using Bayesian inferences to improve quantitative analysis in special education research with qualitative knowledge.

It is impossible, however, to fully disentangle philosophy of inquiry, methodology, and methods. My evaluation of Bayesian estimators, thus, addresses each of these areas. Chapter Two offers a philosophical argument to substantiate the thesis that Bayesian inference is usually …

Validation Of The Scores Of The Instructional Pedagogical And Instructional Student Engagement Components Of Fidelity Of Implementation, Sandra F. Naoom

USF Tampa Graduate Theses and Dissertations

Students cannot benefit from what they do not experience. Multiple reasons exist for why an intervention may not be delivered as it was designed. In this era of educational accountability and limited dollars to go around, understanding how an intervention is delivered in the classroom is key to understanding program outcomes. In order to assess whether a program has been implemented as intended, an assessment of fidelity is needed. However assessing fidelity is complex given varying conceptual interpretations, which then fosters inconsistent application of methods to measure the construct. Additionally the methods for validating fidelity measures are still unclear. The …

Just Love: A Collaborative Evaluation Of A Faith-Based School-Family-Community Partnership Through The Voices Of The Children, Lynette M. Henry

USF Tampa Graduate Theses and Dissertations

Abstract

Faith-based school-family-community partnerships have been a federal mandate over the past decade, yet little has been written about the outcomes of these faith-based partnerships. A need exists to understand if the potential in these faith-based partnerships is indeed realized in positive outcomes for students and schools. The purpose of this study was to evaluate a faith-based school-family-community partnership, Just Love. Just Love is a faith-based school-family-community partnership between a large suburban church and a Title 1, urban elementary school, Charisma Elementary School (Charisma ES; pseudonym). It was implemented in what is considered a "failing school". Just Love's purpose was …

Applying Public Relations Theory To Assess Service-Learning Relationships, Karen Strand

USF Tampa Graduate Theses and Dissertations

In Service-Learning (S-L) partnerships, universities and community organizations exchange resources and influence. Community engagement scholars Cruz and Giles proposed that relationships within S-L partnerships serve as units of analysis for the study of community outcomes of engagement. Yet, the scholarship of engagement lacks a suitable instrument to assess such relationships. This study brings together two lines of scholarship-relationship studies within community engagement and cocreational studies within public relations-to address the problem of assessing the community outcomes of S-L relationships, and it applies Cruz and Giles' ideas about using relationship analysis to assess community outcomes when it considers the perspectives of …

A Monte Carlo Study: The Consequences Of The Misspecification Of The Level-1 Error Structure, Merlande Petit-Bois

USF Tampa Graduate Theses and Dissertations

Single-case interventions allow for the repeated measurement of a case or participant across multiple time points, to assess the treatment¡͞s effect on one specific case or participant. The basic interrupted time series design includes two phases: baseline and treatment. Raudenbush and Byrk (2002) demonstrated that a meta-analysis of large group designs can be seen as a special case of multi-level analysis with participants (level-one) nested within studies (level-two). Raw data from a set of single case design studies have a similar structure. Van den Noortgate and Onghena (2003) illustrated the use of a two-level model to analyze data in primary …

The Performance Of The Linear Logistic Test Model When The Q-Matrix Is Misspecified: A Simulation Study, George T. Macdonald

USF Tampa Graduate Theses and Dissertations

A simulation study was conducted to explore the performance of the linear logistic test model (LLTM) when the relationships between items and cognitive components were misspecified. Factors manipulated included percent of misspecification (0%, 1%, 5%, 10%, and 15%), form of misspecification (under-specification, balanced misspecification, and over-specification), sample size (20, 40, 80, 160, 320, 640, and 1280), Q-matrix density (60% and 46%), number of items (20, 40, and 60 items), and skewness of person ability distribution (-0.5, 0, and 0.5). Statistical bias, root mean squared error, confidence interval coverage, confidence interval width, and pairwise cognitive components correlations were computed. The impact …

The Relationship Between Rating Scales Used To Evaluate Tasks From Task Inventories For Licensure And Certification Examinations, Adrienne W. Cadle

USF Tampa Graduate Theses and Dissertations

The first step in developing or updating a licensure or certification examination is to conduct a job or task analysis. Following completion of the job analysis, a survey validation study is performed to validate the results of the job analysis and to obtain task ratings so that an examination blueprint may be created. Psychometricians and job analysts have spent years arguing over the choice of scales that should be used to evaluate job tasks, as well as how those scales should be combined to create an examination blueprint. The purpose of this study was to determine the relationship between individual …

Value-Added And Observational Measures Used In The Teacher Evaluation Process: A Validation Study, Claudia Güerere

USF Tampa Graduate Theses and Dissertations

Scores from value-added models (VAMs), as used for educational accountability, represent the educational effect teachers have on their students. The use of these scores in teacher evaluations for high-stakes decision making is new for the State of Florida. Validity evidence that supports or questions the use of these scores is critically needed. This research, using data from 2385 teachers from 104 schools in one school district in Florida, examined the validity of the value-added scores by correlating these scores with scores from an observational rubric used in the teacher evaluation process. The VAM scores also were examined in relation to …

Perceptions Of Workplace Mentoring Behaviors For Lifelong Career Development, Lynne A. Key

USF Tampa Graduate Theses and Dissertations

This study's purpose was to investigate the importance of mentoring functions and behaviors for lifelong career development as perceived by protégés. The population included individuals in middle to late adulthood (age 40 years and older) who reported they had been a protégé in at least one mentoring association perceived as beneficial to their lifelong career development; and were either employed or had been employed as a middle manager, senior manager, C-level executive, business owner, or member of a profession. The sample was obtained using a chain-sample method; 67 Ambassadors completed an online survey and each invited 10 contacts to complete …

Effectiveness Of Propensity Score Methods In A Multilevel Framework: A Monte Carlo Study, Aarti P. Bellara

USF Tampa Graduate Theses and Dissertations

Propensity score analysis has been used to minimize the selection bias in observational studies to identify causal relationships. A propensity score is an estimate of an individual's probability of being placed in a treatment group given a set of covariates. Propensity score analysis aims to use the estimate to create balanced groups, akin to a randomized experiment. This study used Monte Carlo methods to examine the appropriateness of using propensity score methods to achieve balance between groups on observed covariates and reproduce treatment effect estimates in multilevel studies. Specifically, this study examined the extent to which four different propensity score …

Translation, Adaptation And Invariance Testing Of The Teaching Perspectives Inventory: Comparing Faculty Of Malaysia And The United States, Jecky Misieng

USF Tampa Graduate Theses and Dissertations

As a result of growing attention in cross-cultural research, existing measurement instruments developed in one language are being translated and adapted for use in other languages and cultural contexts. The benefits of having the same instrument across cultures can only be realized if the process of translation and adaptation of the measurement instruments produces measurement operations that function similarly across national and cultural boundaries. Producing invariant measurement instruments that assess educational and psychological constructs provide a way of testing the cross-cultural generality of theories that include these constructs.

The major purposes of the study were to translate and adapt the …

Reading Assessment Practices Of Elementary General Education Teachers: A Descriptive Study, Sarah Mirlenbrink Bombly

USF Tampa Graduate Theses and Dissertations

In this descriptive study, I researched five elementary general education teachers' reading assessment practices as they worked within the context of IDEA (2004), NCLB (2002) and Response to Intervention (RTI). My own connection to the classroom and reading assessment practices brought me to this research. I presented my personal and professional connection through vignettes about my own classroom assessment practices. Relevant literature on both the context and culture of assessment were pertinent to this research.

I used a qualitative design, specifically, Colaizzi's (1978) method of phenomenological analysis. Data were three in-depth phenomenological interviews, relevant documents and artifacts, and use of …

An Analysis Of Factor Extraction Strategies: A Comparison Of The Relative Strengths Of Principal Axis, Ordinary Least Squares, And Maximum Likelihood In Research Contexts That Include Both Categorical And Continuous Variables, Kevin Barry Coughlin

USF Tampa Graduate Theses and Dissertations

This study is intended to provide researchers with empirically derived guidelines for conducting factor analytic studies in research contexts that include dichotomous and continuous levels of measurement. This study is based on the hypotheses that ordinary least squares (OLS) factor analysis will yield more accurate parameter estimates than maximum likelihood (ML) and principal axis factor anlaysis (PAF); the level of improvement in estimates will be related to the proportion of observed variables that are dichotomized and the strength of communalities within the data sets.

To achieve this study's objective, maximum likelihood, ordinary least squares, and principal axis factor extraction models …

Sensitivity Of Value Added School Effect Estimates To Different Model Specifications And Outcome Measures, Bryce L. Pride

USF Tampa Graduate Theses and Dissertations

The Adequate Yearly Progress (AYP) Model has been used to make many high-stakes decisions concerning schools, though it does not provide a complete assessment of student academic achievement and school effectiveness. To provide a clearer perspective, many states have implemented various Growth and Value Added Models, in addition to AYP. The purpose of this study was to examine two Value Added Model specifications, the Gain Score Model and the Layered Effects Model, to understand similarities and differences in school effect results. Specifically, this study correlated value added school effect estimates, which were derived from two model specifications and two outcome …

An Examination Of The Relationship Between Elementary Education Teacher Candidates' Authentic Assessments And Performance On The Professional Education Subtests On The Florida Teacher Certification Exam (Ftce), Thomas Raymond Lang Ii

USF Tampa Graduate Theses and Dissertations

Abstract

According to Wilkerson and Lang (2003, p.1) with approximately "90% of schools, colleges, and departments of education using portfolios of one form or another as decision-making tools for standards-based decisions regarding certification or licensure (as well as NCATE accreditation), it is appropriate to explore the legal and psychometric aspects of this assessment device."

This study was conducted to examine how well the authentic assessments created in the Chalk and Wire ePortfolio initiative, which was created to provide authentic assessments of the Accomplished Practices, relate to the measures in the Professional Knowledge subtests on Florida Teacher Certification Examination.

The sample …