Education Commons^™

An Empirical Comparison Of The Effect Of Missing Data On Type I Error And Statistical Power Of The Likelihood Ratio Test For Differential Item Functioning: An Item Response Theory Approach Using The Graded Response Model, Patricia Rodriguez De Gil

USF Tampa Graduate Theses and Dissertations

In the context of educational research, missing data arise when examinees omit or do not reach an item, which generates an item nonresponse problem. Using a simulation approach, in addition to conducting complete data analyses, this study compared the performance of six methods for treating item nonresponse in the context of differential item functioning (DIF). The effect of missing data on the Type I error and statistical power of the Likelihood Ratio test for DIF detection in small scales was examined in the context of Item Response Theory (IRT-LR), using polytomous, Likert-type data and the graded response model. The effect …

Numeracy Infusion Course For Higher Education (Niche), 1: Teaching Faculty How To Improve Students' Quantitative Reasoning Skills Through Cognitive Illusions, Frank Wang, Esther I. Wilder

Numeracy

We describe one of the eight units of a professional development program, the Numeracy Infusion Course for Higher Education (NICHE), which introduces research on cognition, including dual-processing theories, to university faculty. Under the dual-processing framework, System 1 (intuition) quickly proposes intuitive answers to judgment problems as they arise, while System 2 (deliberation) monitors the quality of these proposals, which it may endorse, correct, or override. We present several classic questions that demonstrate the pitfalls of overreliance on intuition without analytical thinking, then describe faculty participants’ responses to these questions and their ideas on how to apply cognitive illusion research to …

Quantitative Reasoning In Environmental Science: Rasch Measurement To Support Qr Assessment, Robert L. Mayes, Kent Rittschof, Jennifer H. Forrester, Jennifer D. Schuttlefield Christus, Lisa Watson, Franziska Peterson

Numeracy

The ability of middle and high school students to reason quantitatively within the context of environmental science was investigated. A quantitative reasoning (QR) learning progression, with associated QR assessments in the content areas of biodiversity, water, and carbon, was developed based on three QR progress variables: quantification act, quantitative interpretation, and quantitative modeling. Diagnostic instruments were developed specifically for the progress variable quantitative interpretation (QI), each consisting of 96 Likert-scale items. Each content version of the instrument focused on three scale levels (macro scale, micro scale, and landscape scale) and four elements of QI identified in prior research (trend, translation, …

The Levels Of Conceptual Understanding In Statistics (Locus) Project: Results Of The Pilot Study, Douglas Whitaker, Steven Foti, Tim Jacobbe

Numeracy

The Levels of Conceptual Understanding in Statistics (LOCUS) project (NSF DRL-111868) has created assessments that measure conceptual (rather than procedural) understanding of statistics as outlined in GAISE Framework (Franklin et al., 2007, Guidelines for Assessment and Instruction in Statistics Education, American Statistical Association). Here we provide a brief overview of the LOCUS project and present results from multiple-choice items on the pilot administration of the assessments with data collected from over 3400 students in grades 6-12 across six states. These results help illustrate students’ understanding of statistical topics prior to the implementation of the Common Core State Standards. Using the …

The Quantitative Reasoning For College Science (Quarcs) Assessment, 1: Development And Validation, Katherine B. Follette, Donald W. Mccarthy, Erin Dokter, Sanlyn Buxner, Edward Prather

Numeracy

Science is an inherently quantitative endeavor, and general education science courses are taken by a majority of college students. As such, they are a powerful venue for advancing students’ skills and attitudes toward mathematics. This article reports on the development and validation of the Quantitative Reasoning for College Science (QuaRCS) Assessment, a numeracy assessment instrument designed for college-level general education science students. It has been administered to more than four thousand students over eight semesters of refinement. We show that the QuaRCS is able to distinguish varying levels of quantitative literacy and present performance statistics for both individual items and …

What You Know Counts: Why We Should Elicit Prior Probabilities From Experts To Improve Quantitative Analysis With Qualitative Knowledge In Special Education Science, Tyler Aaron Hicks

USF Tampa Graduate Theses and Dissertations

Qualitative knowledge is about types of things, and their excellences. There are many ways we humans produce qualitative knowledge about the world, and much of it is derived from non-quantitative sources (e.g., narratives, clinical experiences, intuitions). The purpose of my dissertation was to investigate the possibility of using Bayesian inferences to improve quantitative analysis in special education research with qualitative knowledge.

It is impossible, however, to fully disentangle philosophy of inquiry, methodology, and methods. My evaluation of Bayesian estimators, thus, addresses each of these areas. Chapter Two offers a philosophical argument to substantiate the thesis that Bayesian inference is usually …

Assessment For Improvement: Two Models For Assessing A Large Quantitative Reasoning Requirement, Mary C. Wright, Joseph E. Howard

Numeracy

We present two models for assessment of a large and diverse quantitative reasoning (QR) requirement at the University of Michigan. These approaches address two key challenges in assessment: (1) dissemination of findings for curricular improvement and (2) resource constraints associated with measurement of large programs. Approaches we present for data collection include convergent validation of self-report surveys, as well as use of mixed methods and learning analytics. Strategies we present for dissemination of findings include meetings with instructors to share data and best practices, sharing of results through social media, and use of easily accessible dashboards. These assessment approaches may …

Effects Of Reducing The Cognitive Load Of Mathematics Test Items On Student Performance, Susan C. Gillmor, John Poggio, Susan Embretson

Numeracy

This study explores a new item-writing framework for improving the validity of math assessment items. The authors transfer insights from Cognitive Load Theory (CLT), traditionally used in instructional design, to educational measurement. Fifteen, multiple-choice math assessment items were modified using research-based strategies for reducing extraneous cognitive load. An experimental design with 222 middle-school students tested the effects of the reduced cognitive load items on student performance and anxiety. Significant findings confirm the main research hypothesis that reducing the cognitive load of math assessment items improves student performance. Three load-reducing item modifications are identified as particularly effective for reducing item difficulty: …

Assessing Numeracy In The Upper Elementary And Middle School Years, Carol Ann Gittens

Numeracy

Numeracy is the ability or tendency to reason critically about quantitative information. The preponderance of published research on numeracy examines this construct among either pre-K or early elementary samples, students with developmental challenges, or is focused on post-secondary and adult cohorts. The numeracy skills of upper-elementary and middle school students is less documented and understood, most notably because of the lack of valid instruments that are developmentally appropriate for the age range. A numeracy scale for use among upper-elementary and middle school students is introduced in this paper. Scale validation was performed using a gender-balanced, racially / ethnically diverse sample …

Educational Assessment Is An Enduring Theme Of Numeracy, H. L. Vacher

Numeracy

The Assessment Theme Collection in this issue brings the count of papers on QL assessment to 22 out of the 136 papers (16.2%) in the journal’s first 15 issues. After the first ten issues (our first five years), the counts were 13 and 85 respectively (15.1%). A table in this editorial updates the list of our papers on the subject.

Within-Level Group Factorial Invariance With Multilevel Data: Multilevel Factor Mixture And Multilevel Mimic Models, Eun Sook Kim, Myeongsun Yoon, Yao Wen, Wen Luo, Oi-Man Kwok

Educational Measurement and Research Faculty Publications

This study suggests two approaches to factorial invariance testing with multilevel data when the groups are at the within level: multilevel factor mixture model for known classes (ML FMM) and multilevel multiple indicators multiple causes model (ML MIMIC). The adequacy of the proposed approaches was investigated using Monte Carlo simulations. Additionally, the performance of different types of model selection criteria for determining factorial invariance or in detecting item noninvariance was examined. Generally, both ML FMM and ML MIMIC demonstrated acceptable performance with high true positive and low false positive rates, but the performance depended on the fit statistics used for …