Educational Assessment, Evaluation, and Research Commons^™

Institutional Investment In Teaching Excellence, Steven C. Taylor, Catherine Haras, Emily Daniell Magruder, Eva Fernández, Margery Ginsberg, Jake Glover

Publications and Research

No abstract provided.

Should We Think About The Scale When We Weigh The Pig?, Kathy E. Clarke, Gretchen A. Hazard

Libraries

To make a heavier pig, then it makes perfect sense to weigh it – twice – prior to feeding him and afterwards. But what if we more interested in the overall health of the pig, or, if our big is rounder post-food, or if our pig can make pig-fattening food choices? To answer those questions, assessors would need to consider a highly-attuned instrument or one that measures something different than the information a scale provides. In this presentation, we hope to engage participants to consider the how of assessment and in particular consider how we assess influences what we find …

Introduction: History And Conceptual Basis Of Assessment In Higher Education, Peter Ewell, Tammie Cumming

Publications and Research

Assessment and accountability are now inescapable features of the landscape of higher education, and ensuring that these assessments are psychometrically sound has become a high priority for accrediting agencies and therefore also for higher education institutions. Bringing together the higher education assessment literature with the psychometric literature, this book focuses on how to practice sound assessment.

This volume provides comprehensive and detailed descriptions of tools for and approaches to assessing student learning outcomes in higher education. The book is guided by the core purpose of assessment, which is to enable faculty, administrators, and student affairs professionals with the information they …

Shifting The Focus Of Naplan, Geoff N. Masters Ao

Teacher columnist – Geoff Masters

The decision to move NAPLAN online provides an opportunity to place less emphasis on comparing the performances of schools and more emphasis on supporting student learning, according to Professor Geoff Masters AO.

Saudi Science Teachers' Assessment Practices In Alzelfy Educational Administration According To Their Perceptions, Saeed Mohammad Alshamrani

International Journal for Research in Education

This paper aimed to identify science teachers' assessment practices according to their perceptions through identifying the assessment goals, activities, criteria, and the nature of assessment reports (types and data). It used the descriptive method; the population was all science teacher in Alzelfy educational administration in Saudi Arabia (106 teachers); however, the response rate was (38%) of the population. A questionnaire with four dimensions was developed; these dimensions are: the goals (two items), activities (20 items), criteria (six items), and the nature of reports (11 items). The results indicated that the teachers thought that they practiced the assessment highly to achieve …

But Can We Measure It?, Geoff N. Masters Ao

Teacher columnist – Geoff Masters

It’s often asserted that some things can’t be measured, Professor Geoff Masters AO writes in his latest column. But how true is this? And if we can’t measure something, should we stop pretending we can teach or develop it?

Preliminary Evaluation Of The Psychometric Quality Of HeightenTm Quantitative Literacy, Katrina C. Roohr, Hyesun Lee, Jun Xu, Ou Lydia Liu, Zhen Wang

Numeracy

Quantitative literacy has been identified as an important student learning outcome (SLO) by both the higher education and workforce communities. This paper aims to provide preliminary evidence of the psychometric quality of the pilot forms for HEIghten quantitative literacy, a next-generation SLO assessment for students in higher education. We evaluated the psychometric quality of the test items (e.g., item analyses), individual- and group-level reliability, the relationship with student performance and related variables (e.g., grade point average) as well as student perceptions, and differences across college-related and demographic subgroups. Our study used data from a pilot test administered to over 1,500 …

Universal Design For Learning: A New Clinical Practice Assessment Tool Toward Creating Access And Equity For All Students, Diane Fogarty

LMU Theses and Dissertations

To examine to what extent current general education pre-service teachers within a teacher preparation program at a private institution of higher education know and understand the principles of Universal Design for Learning (UDL), expert focus groups were conducted. General education program syllabi were examined for UDL content and found to be lacking in such content. General education pre-service teachers videotaped lessons were reviewed for UDL content and were also found to be inadequate in demonstrating knowledge and understanding of Universal Design for Learning principles. Focus groups comprised of university fieldwork instructors and teacher education experts were asked to review and …

Multi-Choice Questions And Their Problems When Used For Assessment Of Aircraft Engineers Education, Ian R. Mcandrew, Ken L. Witcher, Elena Navarro, Peter Foreman

Publications

Licensed aircraft engineers under the European Aviation Safety Agency, EASA, undertake academic training to complement their practical and type specific studies. These exams are mainly Multi-Choice Questions, MCQ, and four 20-minute essays. The MCQ exams are as few as 16 questions to a maximum of 140 questions. A score of 75% is needed to pass each exam, and each question has three possible answers. This authors of this paper reviews the theory and design of the MCQ and asks if the assumptions are valid and that it achieves the academic level assumed for engineers who will be maintaining some of …

Informal Order, Needs Analysis, And The Eap Curriculum, Faisal S. Al-Maamari

The Qualitative Report

The academic curriculum is developed through a systematic process whereby content is created through the alignment of needs to stakeholder or target group. This qualitative research study features a small-scale, English for academic purpose (EAP) needs analysis (NA) of three credit-bearing EAP programs and the corresponding departmental programs conducted at a Language Center at a higher education institution in Oman. Based on interview, observational and documentary data, the analysis showed divergences in academic literacy (writing and reading) between the EAP and content programs. Principally, the findings pointed to the presence and operation of a group of informal orders and the …

Design And Assessment Of Deep And Active Learning In Science, Technology, Engineering, And Mathematics (Stem) Education, Juhong Christie Liu Ph.D., Elizabeth Johnson Ph.D., Jin Joy Mao Ph.D.

Libraries

This presentation draws academic significance from a focused literature review and initial data for learning design in Science, Technology, Engineering, and Mathematics (STEM) education. The presenters will discuss strategies of fostering deep, active learning, alignment with assessment, and development of assessment instruments and methods. The presenters will share the design and development of an assessment kit to measure learning outcomes that matter in the 21st century STEM education. Through shared evidence and interactive reflection, the audience will take away up-to-date design strategies for deep and active learning as well as assessment in STEM education. The project is sponsored by National …

Assessment Online: Informing Teaching And Learning, Geoff N. Masters Ao

Teacher columnist – Geoff Masters

Online assessments are capable of providing significantly improved feedback to teaching and learning. Experience in schools is demonstrating the potential of online assessment – provided the foundations are right.

Assessment Online: Informing Teaching And Learning, Geoff N. Masters

Teacher columnist – Geoff Masters

Online assessments are capable of providing significantly improved feedback to teaching and learning. Experience in schools is demonstrating the potential of online assessment – provided the foundations are right.

The advantages of online assessment are often described in terms of its administrative convenience, efficiency and lower costs. However, well-constructed online assessments also are capable of providing more timely, more instructionally useful feedback to teaching and learning. For a number of years ACER has been investigating ways to enhance the educational value of online assessments.

The Introduction Of Informal Cooperative Learning Into Our Programming Laboratories, Guity Ravai, Ludmila Nunes, Ronald Erdei

IMPACT Presentations

Presented at the Women in Engineering ProActive Network (WEPAN) Change Leader Forum: Creating a Mindset for Action in Westminster, CO, USA

What Impacts Do Oer Have On Students? Students Share Their Experiences With A Health Psychology Oer At New York City College Of Technology, Cailean Cooney

Publications and Research

This article reports findings from a study conducted with students in three sections of a Health Psychology course that replaced a traditional textbook with open educational resources (OER) as the primary course material. The purpose of the study was to learn how OER impacted students. Data were collected in Fall 2015 with students from New York City College of Technology (City Tech), of the City University of New York (CUNY), a comprehensive college located in Brooklyn. Students were assigned the OER by their course instructor, who developed it as part of a library funded OER pilot initiative. Two research instruments …

Using Concept Maps To Monitor Knowledge Structure Changes In A Science Classroom, Leah J. Cook

Dissertations

The aim of this research is to determine what differences may exist in students’ structural knowledge while using a variety of concept mapping assessments. A concept map can be used as an assessment which connects concepts in a knowledge domain. A single assessment may not be powerful enough to establish how students’ new knowledge relates to prior knowledge. More research is needed to establish how various aspects of the concept mapping task influence the output of map creation by students. Using multiple concept maps and pre-instruction and postinstruction VNOS instruments during a 16-week semester, this study was designed to investigate …

Predicting Teacher Value-Added Results In Non-Tested Subjects Based On Confounding Variables: A Multinomial Logistic Regression, Nathan Street

Doctoral Dissertations and Projects

Teacher value-added measures (VAM) are designed to provide information regarding teachers’ causal impact on the academic growth of students while controlling for exogenous variables. While some researchers contend VAMs successfully and authentically measure teacher causality on learning, others suggest VAMs cannot adequately control for exogenous influences on the classroom. Furthermore, because VAMs are primarily connected to student performance on standardized, high-stakes exams and those exams are resoundingly considered to be inadequate measures of true student learning, educators and educational leaders assert VAM results are moot. The purpose of this study was to consider the potential for student background, teacher preparation, …

Exposing Problems Teaching Students Morphological Species Identification, Trevor Stamper, Lauren Weidner, Gregory Nigoghosian, Ludmila D. Nunes

IMPACT Presentations

When dealing with physical remains, morphological assessment for species is a traditional approach to entomological specimen identification. A dichotomous key guides the user through taxa determination for a specimen by providing a series of dual-choice nodes that center around morphological differences. Each nodal choice leads to either a new set of dichotomous choices or a taxa decision. We evaluated student’s ability to utilize a dichotomous key down to species for a limited set of taxa, by reviewing their nodal decisions along with their confidence level using a Likert scale (1-5).

Along with individual decision recording, students conducted a post-decision group …

Stitching Codeable Circuits: High School Students' Learning About Circuitry And Coding With Electronic Textiles, Breanne Krystine Litts, Yasmin B. Kafai, Debora A. Lui, Justice T. Walker, Sari A. Widman

Instructional Technology and Learning Sciences Faculty Publications

Learning about circuitry by connecting a battery, light bulb, and wires is a common activity in many science classrooms. In this paper, we expand students’ learning about circuitry with electronic textiles, which use conductive thread instead of wires and sewable LEDs instead of lightbulbs, by integrating programming sensor inputs and light outputs and examining how the two domains interact.We implemented an electronic textiles unit with 23 high school students ages 16–17 years who learned how to craft and code circuits with the LilyPad Arduino, an electronic textile construction kit. Our analyses not only confirm significant increases in students’ understanding of …

Pisa 2015: Financial Literacy In Australia, Sue Thomson, Lisa De Bortoli

OECD Programme for International Student Assessment (PISA) Australia

The main focus of the Programme for International Student Assessment (PISA) financial literacy assessment is on measuring the proficiency of 15-year-old students in demonstrating and applying the knowledge and skill that they have learned in and out of school. Like other PISA domains, financial literacy is assessed using an instrument designed to provide data that are valid, reliable and interpretable. PISA is an international comparative assessment of student achievement directed by the Organisation for Economic Co-operation and Development (OECD). PISA measures how well 15-year-olds, who are nearing the end of their compulsory schooling in most participating educational systems, are prepared …

Argument Education In Higher Education: A Validation Study, Paul E. Mabrey Iii

Dissertations, 2014-2019

Argument education can play an important role in higher education for leadership development and responding to increasing calls for post-secondary accountability. But to do so, argumentation teachers, scholars, and practitioners need to develop a clearer definition and research agenda for the purposes of teaching and assessing argumentation. The research conducted here contributes to this project by first establishing a definitional construct and observable behaviors associated with learning and practicing argumentation. Second, an argument education assessment instrument was created based off of the literature-supported definition of argumentation. Third, debate and argument education subject matter experts reviewed the definition, behaviors, and assessment …

The Consequential Validity Of The M-Step And Third-Grade Retention, Michelle L. Sprouse

Language Arts Journal of Michigan

This paper applies Slomp, Corrigan, and Sugimoto’s (2014) consequential validity framework to the third-grade Michigan Student Test of Educational Progress (M-STEP) in English language arts (ELA). Slomp et al.’s (2014) consequential validity framework provides for a holistic examination of the validity of the assessment and its consequences. Using this framework, this paper considers the construct of reading developed in the adopted standards, assessment design, sample assessment items, disaggregated performance data, and the assessment consequences. The number and magnitude of validity concerns raised in all aspects of the framework call into question the consequential validity of the assessment.

How Well Are We Learning From Naplan?, Geoff N. Masters Ao

Teacher columnist – Geoff Masters

Some Australian schools and school systems have seen greater improvements in NAPLAN results than others. How well do we understand where improvements are occurring and why? Professor Geoff Masters AO discusses.

Teaching, Learning, And Assessment: Insights Into Students’ Motivation To Learn, Simon R. Walters, Pedro Silva, Jennifer Nikolai

The Qualitative Report

This study draws upon the perspectives of sport and recreation undergraduate students in New Zealand who were involved in the design of their own assessments, and discusses the implication of the teaching and learning environment on this process. In a previous study, student criticism had emerged of current teaching strategies and assessment methods at their institution. The purpose of this current study was to directly address some of these concerns and for lecturers and students to work collaboratively to develop a more learner-centred teaching and learning environment. Students from a second-year sociology of sport paper were invited to design their …

Using Educative Assessment In A Science Classroom, Jordan Truitt

Honors Projects

The focus of this study is to consider educative assessment in the science classroom and its effectiveness in assessing student-centered learning. By using educative assessment to measure learning, student work can be evaluated by rubric in addition to a traditional summative assessment. In comparing the student learning measured using the rubric results and traditional results, there was no statistical difference between the two assessments. However, the educative assessment provided greater insight into higher levels of learning. Conversational data also showed evidence of skills-based learning in the field of science. Further research into the applications for ‘at-risk’ students is required.

The Introduction Of Informal Cooperative Learning Into Our Programming Laboratories, Guity Ravai, Ronald Erdei, Ludmila Nunes, Sahithya Kodam

IMPACT Presentations

Presentation given at the Midwest SoTL Conference. Discusses the effects of implementing informal cooperative learning techniques in an introductory programming course.

Redesigning Field Supervisor Evaluations Based On Results Of A Factor Analysis, Amy Scott Brown, Justin Low, Amy Burns

Pacific's Annual Assessment Conference

No abstract provided.

Multi-Year Improvements In Campus Safety Based On Assessment Results, Mike Belcher

Pacific's Annual Assessment Conference

No abstract provided.

From Isolation To Connection: Assessing The Effectiveness Of An Intervention Model For Students With Social/Communication Challenges, Daniel Nuss, Kim Derushia

Pacific's Annual Assessment Conference

No abstract provided.

Leadership Identity And Critical Thinking Development In Cip Cohort Projects, Allison Dumas

Pacific's Annual Assessment Conference

No abstract provided.