Education Commons^™

Instructional Decision Making In A Gateway Quantitative Reasoning Course, Deependra Budhathoki, Gregory D. Foley, Stephen Shadik

Numeracy

Many educators and professional organizations recommend Quantitative Reasoning as the best entry-level postsecondary mathematics course for non-STEM majors. However, novice and veteran instructors who have no prior experience in teaching a QR course often express their ignorance of the content to choose for this course, the instruction to offer students, and the assessments to measure student learning. We conducted a case study to investigate the initial implementation of an entry-level university quantitative reasoning course during fall semester, 2018. The participants were the course instructor and students. We examined the instructor’s motives and actions and the students’ responses to the course. …

Threshold Concepts In Quantitative Reasoning, Judith Canner, Jennifer E. Clinkenbeard

Numeracy

The idea of “threshold concepts” has been used to identify discipline-based concepts that are critical to that academic area. Threshold concepts are often difficult for students to assimilate in a meaningful way but, once done, can be powerful for the learner. In general, threshold concepts are 1) transformative to learner thinking; 2) bounded by the discipline; 3) integrative with other concepts; and 4) irreversible once understood (Meyer and Land 2003). This paper presents five threshold concepts in quantitative reasoning (QR) developed by transdisciplinary faculty workgroups that may be applicable for non-mathematics disciplines as well. They are as follows: 1) QR …

Grades Or No Grades? Promoting Deeper Learning In A Middle Level Mathematics Methods Course, Shelli L. Casler-Failing

Journal of Practitioner Research

This action research investigated pre-service teachers (PSTs) experiences with ungrading practices in a mathematics methods course designed for middle level PSTs (grades 4-8). This study analyzed archival data through the lenses of pedagogical content knowledge, growth mindset, and self-efficacy to investigate how PSTs’ experiences with the process of ungrading supported their development of pedagogical content knowledge. Throughout the course, verbal feedback was provided during class discourse and the interactive lectures and written feedback was provided for all submitted assignments. The feedback provided was both positive and constructive in nature. Based on the assignment or activity, constructive feedback was either provided …

Establishing The Validity And Reliability Of The Locus Assessments, Tim Jacobbe, Bob Delmas, Brad Hartlaub, Jeff Haberstroh, Catherine Case, Steven Foti, Douglas Whitaker

Numeracy

The development of assessments as part of the funded LOCUS project is described. The assessments measure students’ conceptual understanding of statistics as outlined in the GAISE PreK–12 Framework. Results are reported from a large-scale administration to 3,430 students in grades 6 through 12 in the United States. Items were designed to assess levels of understanding as well as components of the statistical problem solving process as articulated in the GAISE framework. We discuss details of how the model used to develop the LOCUS assessments guided the gathering of evidence for validity and reliability arguments. Three types of validity evidence are …

Focused On Pedagogy: Qr Grading Rubrics For Written Arguments, Ruby Daniels, Kathryn Appenzeller Knowles, Emily Naasz, Amanda Lindner

Numeracy

Institutional assessments of quantitative literacy/reasoning (QL/QR) have been extensively tested and reported in the literature. While appropriate for measuring student learning at the programmatic or institutional level, such instruments were not designed for classroom grading. After modifying a widely accepted institutional rubric designed to assess QR in written arguments, the current mixed method study tested the reliability of two QR analytic grading rubrics for written arguments and explored students’ reactions to the grading tools. Undergraduate students enrolled in a business course (N = 59) participated. A total of 415 QR artifacts from 40 students were assessed; an additional 19 …

Rubrics That Systematically Identify Areas For Improvement, Judy R. Wilkerson, W. Steve Lang

University of South Florida (USF) M3 Publishing

Commonly used analytic and holistic formats for rubrics have significant limitations in their potential to yield meaningful feedback to students. This inhibits students’ potential to improve their work. Consistent with the recommendation of the Gordon Commission for the Future of Assessment in Education that the focus of assessment shift from accountability to improvement, this research presents one easily-implemented step toward that goal. An alternative rubric format that is designed to provide meaningful feedback to students in order to trigger necessary learning and improvement is illustrated, and feedback from students on its utility is documented. The alternative rubric format, which focuses …

Teaching For Knowledge Transfer: Best Practices From A Graduate-Level Educational Psychology Distance Learning Program, Bobby Hoffman

University of South Florida (USF) M3 Publishing

One measure of effective instruction is the ability to solve authentic real-world problems by effectively transferring and applying classroom and textbook knowledge. While many students can productively earn high grades and learn course content, they are not always able to apply the knowledge they gain. As such, this quasi-experimental study compared the comprehensive exit exam results of learners across instructional modalities who completed a prominent graduate-level educational psychology program. ANCOVA revealed superior knowledge transfer for blended-learning students compared to those who completed distance education and significantly greater transfer of declarative, procedural, and self-regulatory knowledge by the blended-learning students. This paper …

Investigating Alignment In A Quantitative Literacy Course For Social Sciences Students, Vera Frith, Pam Lloyd

Numeracy

The Numeracy Centre at the University of Cape Town has taught a one-semester quantitative literacy course for social sciences students since 1999. This study aims to provide an example for how the design of such a course can be assessed for alignment with quantitative reasoning goals. We propose a framework of learning outcomes for the course and use that framework to analyse the assessments and student performance on them. We find that just under half of the overall mark for the course was devoted to the interpretation and communication of quantitative information (our “main” outcomes), and about a quarter was …

Support For Teacher Candidates And The Edtpa, Dia Gary, Dylan Thomas, Joseph Miller

Journal of Global Education and Research

Equipping new teachers in today’s society is a noble and challenging task. Of late, many additional licensing standards for teachers create additional responsibilities for universities that provide pedagogy, knowledge, and content for teacher candidates. Identification of best practices that support teacher candidates on the Educative Teacher Performance Assessment (edTPA) is of interest to many teacher education programs. The purpose of this research was to analyze whether incorporating a mock edTPA had beneficial results on the passage of the edTPA. This study spanned fourteen ten-week quarters and included 688 teacher candidates who were preparing to become certificated teachers at a university …

Probing For Bias: Comparing Populations Using Item Response Curves, Paul J. Walter, Edward Nuhfer, Crisel Suarez

Numeracy

We introduce an approach for making a quantitative comparison of the item response curves (IRCs) of any two populations on a multiple-choice test instrument. In this study, we employ simulated and actual data. We apply our approach to a dataset of 12,187 participants on the 25-item Science Literacy Concept Inventory (SLCI), which includes ample demographic data of the participants. Prior comparisons of the IRCs of different populations addressed only two populations and were made by visual inspection. Our approach allows for quickly comparing the IRCs for many pairs of populations to identify those items where substantial differences exist. For each …

Measuring Numeracy: Validity And The Programme For The International Assessment Of Adult Competencies (Piaac), Samuel L. Tunstall

Numeracy

A tension raised in recent scholarship is that between numeracy as a social practice and numeracy as a functional skill set. Such frameworks for conceptualizing numeracy pose a challenge to assessment because what individuals do with numeracy is not the same as what individuals can do (or express) in an assessment setting. This study builds on work related to numeracy assessment through a validity examination of a portion of a well-known assessment: the OECD’s Programme for the International Assessment of Adult Competencies (PIAAC). In following a path set out by standards for assessment, I ask: What does the PIAAC numeracy …

Are We At A Watershed Moment For The Quantitative Literacy Movement?: Review Of Shifting Context, Stable Core: Advancing Quantitative Literacy In Higher Education, By Luke Tunstall, Gizem Karaali, And Victor Piercey, Eds., Maura Mast

Numeracy

Luke Tunstall, Gizem Karaali, and Victor Piercey, eds. 2019. Shifting Concepts, Stable Core: Advancing Quantitative Literacy in Higher Education. Math Notes 88. (Mathematics Association of America, MAA Press). Print ISBN 978-0-88385-198-2. Electronic ISBN 978-1-61444-324-7.

The thematic approach of the edited MAA Notes volume Shifting Contexts, Stable Core: Advancing Quantitative Literacy in Higher Education is that the “construct” of quantitative literacy is now fairly stable, but the contexts in which quantitative literacy is taught (and practiced) continue to change. Several chapters give the reader much to consider regarding what constitutes the foundation of this stable core and, relatedly, how quantitative …

Alignment Between Learning Objectives And Assessments In A Quantitative Literacy Course, Younggon Bae, Samuel L. Tunstall, Kathryn S. Knowles, Rebecca L. Matz

Numeracy

In this analysis, we examine how course assessment items were aligned with learning objectives in a quantitative literacy course at Michigan State University. The alignment analysis consisted of mapping assessment items to a list of operationalized learning objectives from the course. Our analysis shows how often the learning objectives are represented in assessment items, how often they are paired with other learning objectives, and how influential they are in contributing to a student’s course grade. In addition, through comparisons across four assessment types (e.g., exams and homework), we show how each learning objective was assessed differently within each assessment type. …

Paired Measures Of Competence And Confidence Illuminate Impacts Of Privilege On College Students, Rachel M. Watson, Edward Nuhfer, Kali Nicholas Moon, Steven Fleisher, Paul Walter, Karl Wirth, Christopher Cogan, Ami Wangeline, Eric Gaze

Numeracy

We seek to understand how the experiences of groups that differ in gender, ethnicity, and sexual orientation produce college-level educational performances that differ from the experiences of the dominant majority group. We employ two datasets: a National Database of 24,701 participants and a Paired-Measures Database with 3,323 participants. Both datasets provide demographic information, socioeconomic conditions of status as first-generation student, English as a first language, and interest in majoring in science, and competency scores on understanding science as a way of knowing obtained from the Science Literacy Concept Inventory. The Paired-Measures Database includes additional self-assessed competence ratings that enabled quantifying …

Quantitative Literacy In The Core Curriculum Of Hood College: Chapter Ii, Outcomes And Assessment, Betty Mayfield, Ann Stewart

Numeracy

In a previous article, we described our college’s new core curriculum, which included a Quantitative Literacy (QL) component for the first time. We explained how we defined QL in the college catalog, and how we used that definition to choose courses to satisfy the new requirement. We then discussed our early efforts at assessing the effectiveness of the QL program and described our plans for the future. Here we report on our progress towards those goals, including working with faculty from other departments and with our institutional research office to develop a more sophisticated assessment plan, as well as creating …

Using The Quantitative Literacy And Reasoning Assessment (Qlra) For Early Detection Of Students In Need Of Academic Support In Introductory Courses In A Quantitative Discipline: A Case Study, Nathan D. Grawe, Kristin O'Connell

Numeracy

As the number of young people attending college has increased, the diversity of college students’ educational backgrounds has also risen. Some students enter introductory courses with math anxiety or gaps in their quantitative training that impede their ability to master or even grasp relevant disciplinary content. Too often professors learn of these anxieties and gaps only during the post mortem of the first midterm. By that time, a good portion of a student’s grade is determined and successful recovery may be impossible. During the 2016-17 academic year, the Department of Economics at Carleton College ran a pilot project using the …

Preliminary Evaluation Of The Psychometric Quality Of HeightenTm Quantitative Literacy, Katrina C. Roohr, Hyesun Lee, Jun Xu, Ou Lydia Liu, Zhen Wang

Numeracy

Quantitative literacy has been identified as an important student learning outcome (SLO) by both the higher education and workforce communities. This paper aims to provide preliminary evidence of the psychometric quality of the pilot forms for HEIghten quantitative literacy, a next-generation SLO assessment for students in higher education. We evaluated the psychometric quality of the test items (e.g., item analyses), individual- and group-level reliability, the relationship with student performance and related variables (e.g., grade point average) as well as student perceptions, and differences across college-related and demographic subgroups. Our study used data from a pilot test administered to over 1,500 …

The Quantitative Reasoning For College Science (Quarcs) Assessment 2: Demographic, Academic And Attitudinal Variables As Predictors Of Quantitative Ability, Katherine Follette, Sanlyn Buxner, Erin Dokter, Donald Mccarthy, Beau Vezino, Laci Brock, Edward Prather

Numeracy

In this article, we explore the ability of demographic and attitudinal variables to predict student scores on the Quantitative Reasoning for College Science (QuaRCS) Assessment. Variables measured by the assessment include: students' academic choices and plans, attitudes and perceptions regarding mathematics, self-reported effort level, and basic demographics such as age, race/ethnicity, gender and disability status. As in previously published numeracy studies, we find significant score deviations according to gender, race/ethnicity, and disability status; however, the effect size of these correlations pale in comparison to the effect size of affective/attitudinal variables on QuaRCS score. A large number of variables with …

The Levels Of Conceptual Understanding In Statistics (Locus) Project: Results Of The Pilot Study, Douglas Whitaker, Steven Foti, Tim Jacobbe

Numeracy

The Levels of Conceptual Understanding in Statistics (LOCUS) project (NSF DRL-111868) has created assessments that measure conceptual (rather than procedural) understanding of statistics as outlined in GAISE Framework (Franklin et al., 2007, Guidelines for Assessment and Instruction in Statistics Education, American Statistical Association). Here we provide a brief overview of the LOCUS project and present results from multiple-choice items on the pilot administration of the assessments with data collected from over 3400 students in grades 6-12 across six states. These results help illustrate students’ understanding of statistical topics prior to the implementation of the Common Core State Standards. Using the …

The Quantitative Reasoning For College Science (Quarcs) Assessment, 1: Development And Validation, Katherine B. Follette, Donald W. Mccarthy, Erin Dokter, Sanlyn Buxner, Edward Prather

Numeracy

Science is an inherently quantitative endeavor, and general education science courses are taken by a majority of college students. As such, they are a powerful venue for advancing students’ skills and attitudes toward mathematics. This article reports on the development and validation of the Quantitative Reasoning for College Science (QuaRCS) Assessment, a numeracy assessment instrument designed for college-level general education science students. It has been administered to more than four thousand students over eight semesters of refinement. We show that the QuaRCS is able to distinguish varying levels of quantitative literacy and present performance statistics for both individual items and …

Assessment For Improvement: Two Models For Assessing A Large Quantitative Reasoning Requirement, Mary C. Wright, Joseph E. Howard

Numeracy

We present two models for assessment of a large and diverse quantitative reasoning (QR) requirement at the University of Michigan. These approaches address two key challenges in assessment: (1) dissemination of findings for curricular improvement and (2) resource constraints associated with measurement of large programs. Approaches we present for data collection include convergent validation of self-report surveys, as well as use of mixed methods and learning analytics. Strategies we present for dissemination of findings include meetings with instructors to share data and best practices, sharing of results through social media, and use of easily accessible dashboards. These assessment approaches may …

Assessing Numeracy In The Upper Elementary And Middle School Years, Carol Ann Gittens

Numeracy

Numeracy is the ability or tendency to reason critically about quantitative information. The preponderance of published research on numeracy examines this construct among either pre-K or early elementary samples, students with developmental challenges, or is focused on post-secondary and adult cohorts. The numeracy skills of upper-elementary and middle school students is less documented and understood, most notably because of the lack of valid instruments that are developmentally appropriate for the age range. A numeracy scale for use among upper-elementary and middle school students is introduced in this paper. Scale validation was performed using a gender-balanced, racially / ethnically diverse sample …

Educational Assessment Is An Enduring Theme Of Numeracy, H. L. Vacher

Numeracy

The Assessment Theme Collection in this issue brings the count of papers on QL assessment to 22 out of the 136 papers (16.2%) in the journal’s first 15 issues. After the first ten issues (our first five years), the counts were 13 and 85 respectively (15.1%). A table in this editorial updates the list of our papers on the subject.

Does Completion Of Quantitative Courses Predict Better Quantitative Reasoning-In-Writing Proficiency?, Nathan D. Grawe

Numeracy

Using data from Carleton College, this study explores the connection between students’ completion of a range of quantitative courses and the quality of their quantitative reasoning in writing (QRW) as exhibited in courses throughout the undergraduate curriculum during the first two years of college. Because the assessment takes place in the context of a campus-wide initiative which has improved QRW on the whole, the study identifies course-taking patterns which predict stronger than average improvement. Results suggest QRW is not exceptionally improved by taking courses in statistics, principles of economics, or in the social sciences more broadly. QRW performance is, on …

Parts Of The Whole: Learn More, Learn Better, Dorothy Wallace

Numeracy

Building on previous columns in Numeracy, this column analyzes various teaching techniques in terms of their ability to build cognitive schema, extend existing schema, reinforce learning, move mean understanding of a group of students, and reduce variance in understanding of a group. We offer a pedagogical cycle as an example of how to address multiple learning goals using common teaching methods.

Motivation For Achievement And Attitudes Toward Mathematics Instruction In A Required Calculus Course At The Norwegian University Of Science And Technology, Donna Sundre, Carol Barry, Vidar Gynnild, Erin Tangen Ostgard

Numeracy

This study from the Norwegian University of Science and Technology (NTNU) examines students’ learning goals and attitudes toward mathematics in a first-year calculus course in undergraduate engineering education. Achievement motivation research using the Achievement Goal Questionnaire (AGQ) is advanced from current literature with two additions: (1) a course specific context using introductory college calculus students, and (2) participation of Norwegian students.

Pre- and posttest measures of attitudes indicate that students do change learning goals over time, unfortunately opposite to the instructors’ aspirations. A significant increase in “Mastery Avoidance” and “Work Avoidance” was accompanied with a drop in “Mastery Approach” and …

Numeracy, Financial Literacy, And Financial Decision-Making, Annamaria Lusardi

Numeracy

Financial decisions, be they related to asset building or debt management, require the capacity to do calculations, including some complex ones. But how numerate are individuals, in particular when it comes to calculations related to financial decisions? Studies and surveys implemented in both the United States and in other countries that are described in this paper show the level of numeracy among the population to be very low. Moreover, lack of numeracy is not only widespread but is particularly severe among some demographic groups, such as women, the elderly, and those with low educational attainment. This has potential consequences for …

Parts Of The Whole: An Algebra Lesson, Dorothy Wallace

Numeracy

This column draws on research of Eon Harper to demonstrate how an understanding of his proposed stages of algebra acquisition would inform a systemic overhaul of algebra education. Harper's stages also explain why students may pass a series of algebra courses yet still be unable to make sense of calculus, as well as offering insight on what aspects of algebra support quantitative literacy.

Quantitative Reasoning In The Contemporary World, 3: Assessing Student Learning, Stuart Boersma, Caren Diefenderfer, Shannon W. Dingman, Bernard L. Madison

Numeracy

In this third paper in a series describing the Quantitative Reasoning in the Contemporary World course, the authors provide an adaptation of the Association of American Colleges and Universities quantitative literacy VALUE rubric. Describing achievement levels in six core competencies (interpretation, representation, calculation, analysis/synthesis, and communication), the resulting Quantitative Literacy Assessment Rubric (QLAR) is applicable to grading student work and has exhibited a high degree of reliability in two separate scoring tests (97% and 88% respectively). The distribution of the six core competencies across the 24 case studies in the authors’ quantitative reasoning casebook shows that interpretation, calculation, and analysis/synthesis …

Quantitative Literacy At Michigan State University, 2: Connection To Financial Literacy, Dennis Gilliland, Vince Melfi, Alla Sikorskii, Edward Corcoran, Eleanor Melfi

Numeracy

The lack of capability of making financial decisions has been recently described for the adult United States population. A concerted effort to increase awareness of this crisis, to improve education in quantitative and financial literacy, and to simplify financial decision-making processes is critical to the solution. This paper describes a study that was undertaken to explore the relationship between quantitative literacy and financial literacy for entering college freshmen. In summer 2010, incoming freshmen to Michigan State University were assessed. Well-tested financial literacy items and validated quantitative literacy assessment instruments were administered to 531 subjects. Logistic regression models were used to …