Educational Assessment, Evaluation, and Research Commons^™

Improving Assessments And Evaluations Of Pilots In Collegiate Aviation Programs, Peter Matthew Dittmer Ed.D.

National Training Aircraft Symposium (NTAS)

The purpose of the proposed study is to improve assessments and evaluation techniques of student pilots in collegiate aviation programs. Proper evaluation and assessment of student pilot performance in collegiate aviation programs is important to the success and credibility of the training program. Students in pilot training programs are scrutinized unlike other students in other academic programs. There exists well-defined program evaluation cycles, specific skills and knowledge to be tested, objective criteria on which to judge performance, tight restrictions on evaluator qualifications, and a well-defined curriculum for creating an evaluation context.

This researcher would look at three reasons student pilots …

Improving Academic Learning And Success Through Transparency: A Student’S Perspective, Christina Ouellette, Joanna Boeing Bratton, Ruth C. Slotnick

Core Assessment Project Reports and Publications

No abstract provided.

When Your Plate Is Already Full: Efficient And Meaningful Outcomes Assessment For Busy Law Schools, Melissa N. Henke

Law Faculty Scholarly Articles

The American Bar Association (ABA) accreditation standards involving outcome-based assessment are a game changer for legal education. The standards reaffirm the importance of providing students with formative feedback throughout their course of study to assess and improve student learning. The standards also require law schools to evaluate their effectiveness, and to do so from the perspective of student performance within the institution’s program of study. The relevant question is no longer what are law schools teaching their students, but instead, what are students learning from law schools in terms of the knowledge, skills, and values that are essential for those …

Supporting Examination Reform In Nepal, Mee Young Han

Teacher India

ACER is helping to ensure that school assessment in Nepal focuses on testing higher-order thinking skills.

2020 Nsse, Southwestern Oklahoma State University

Assessment Center Reports

National Survey of Student Engagement (NSSE) reports produced by the SWOSU Assessment Center.

The Viability Of Simulated Large-Scale Marking As Professional Development For Preservice Teachers, Nathanael Reinertsen

Theses: Doctorates and Masters

Judging the quality of student work is a core skill of a proficient teacher. This professional competency is often utilised by organisations that run large-scale marking operations when they recruit teachers as markers. These organisations and the teachers themselves often claim that large-scale marking is valuable professional development.

This research aimed to determine whether professional learning outcomes similar to those reported by experienced teachers can be achieved for preservice teachers through participation in a live simulation of a large-scale marking operation. The research was conducted in three phases: an online survey of Australian teachers to establish that reports of benefit …

Alignment Or Detachment: College Educators’ Perceptions Of Experiential Learning Pedagogy, Wayne D. Finseth

All Graduate Theses, Dissertations, and Other Capstone Projects

The main purpose of this study was to investigate how college educators’ perceptions of experiential learning influence the design, implementation, and assessment of their courses. A second purpose of this study was to examine the way’s college educators use experiential learning in their classes. A final purpose of this study was to evaluate the relationship between a college educators’ prior experience with experiential learning and how they utilize that prior experiences to design, instruct, and assess their courses. A descriptive research design was used in this study. A randomized and anonymous survey containing 29 items was emailed out through Qualtrics …

Occupational Therapy Students’ Perceptions Of Osce: A Qualitative Descriptive Analysis, Nancy E. Krusen, M. Nicole Martino

Journal of Occupational Therapy Education

Objective Structured Clinical Examinations (OSCE) are commonly used across health professions educational programs to evaluate student clinical competencies. OSCE are multiple, brief stations representing common practice scenarios. The purpose of the study was to evaluate student perceptions of OSCE. The researchers implemented 17 OSCE stations with 40 second year occupational therapy students to assess clinical competencies prior to fieldwork. Applying a qualitative descriptive methodologic approach, researchers analyzed station rating data, Qualtrics survey Likert-type items, and Qualtrics survey open-ended responses. Number of station rating responses varied widely, due to perceived time press. Station rating responses confirmed the more robust 80% response …

Developing Text-Based Writing And Working Through The Revision Process, Valerie Clark

Masters Theses

A narrative continuation is designed to assess a student’s ability to comprehend a text selection and use their own creativity to redesign the text’s conclusion. This study took a narrative continuation writing assessment and analyzed the process which 41 sixth grade students followed, specifically looking at the prewriting and revision procedures. The goal was to determine if the supports and resources provided were effective for the type of writing and student population. Throughout the process, students’ behaviors which indicated motivation levels were noted (participation in discussions, level of focus, attentiveness, and amount of redirection required). Student work was then reviewed …

Assessing Teacher Pedagogical Knowledge: The Video Assessment Of Teacher Knowledge (Vatk), Peter D. Wiens, Jori S. Beck, Christina J. Lunsmann

Teaching & Learning Faculty Publications

This study examines the creation and implementation of a video-based assessment of teacher knowledge of effective teaching called the Video Assessment of Teacher Knowledge (VATK). The VATK was developed with the intention of creating an easily administered, standardised assessment that can potentially provide information on teacher candidate knowledge of teaching at important points during their training programmes and into the teaching field. The theoretical framework upon which the VATK was developed and the process for item creation are described. Classical test theory and item response theory analysis were conducted to determine the measure’s potential for use in future studies. Analyses …

Multiple Intelligence Assessment In Teaching English For Young Learners, Ernawati Ernawati, Hana Tsurayya, Abdul Rahman A. Ghani

REID (Research and Evaluation in Education)

There are many schools in Indonesia that provide English as one of their subjects. English has been taught from elementary schools, even in Kindergarten. Ironically, teaching English in most rural schools still uses a conventional method such as memorizing and translating. Many teachers cannot afford to provide well-designed, meaningful exercises for students to use on a one-to-one learning basis. As a result, students seem not having interest in learning English. Based on this reason, this study was conducted to identify students' intelligence through Multiple Intelligence Assessment to get effective approach in teaching English for young learners. The participants are an …

Infographic: Pisa 2018 – Australian Results By State And Territory, Jo Earp

Teacher infographics

The OECD's Programme for International Student Assessment (PISA) measures how well 15-year-olds are prepared to use their skills and knowledge to meet real-life challenges. The 2018 data show Australian student performance in all three assessment domains is in long-term decline. This infographic looks at the low and high performers by jurisdiction.

Infographic: Pisa 2018 – Australian Trends And Comparison, Jo Earp

Teacher infographics

Results from the OECD's 2018 Programme for International Student Assessment (PISA) have been released. The data show Australian student performance is in long-term decline. This infographic looks at Australia’s PISA performance over time, and how the 2018 results compare internationally.

Teacher Perceptions Of Assessments In Character Education: A Case Study, Sarah Hickman

Senior Honors Theses

Since the time of Aristotle, educators have emphasized character as a necessary part of a student’s education, and currently, many states mandate character education by law. Because of this historical and legal emphasis, there is a growing discussion on the necessity of assessments in character education to ensure that character education programs are effective. While there is research on the large-scale effectiveness of programs with different assessments, there is little research about how teachers perceive assessments and measurements in character education. This study was conducted to begin to address this gap in the research. Through the research design of an …

The Assessment Of Faith And Learning, Beth Green, Albert Cheng, David Smith

Education Reform Faculty and Graduate Students Publications

The Practicing Faith Survey (PFS) is a new assessment tool designed to measure the extent to which schoolchildren connect their faith to learning. This paper reviews the landscape of educational assessment and argues that assessment remains a critical element in the design of Christian teaching and learning. It suggests that unease around the concept of educational measurement leads to limited attempts to assess faith formation in the context of learning. The paper discusses PFS as a way to reframe the design process consistent with distinctively Christian practices of teaching and learning.

The Development And Validation Of The Practicing Faith Survey, Albert Cheng, Beth Green, David Smith

Education Reform Faculty and Graduate Students Publications

The Practicing Faith Survey (PFS) is a new assessment tool designed to measure the extent to which schoolchildren connect their faith to learning. PFS measures student engagement with five domains of Christian practice in connection with learning: intellectual, relational, introspective, benevolence, and formational practices. We describe the item-development process and then present evidence for the validity and reliability of the PFS based on a sample of 1,300 fifth- through twelfth-grade students who participated in a pilot of the instrument.

How Schools Create Healthy Testing Cultures And How Principals Use Standardized Test Data To Become Leaders Of Learning, Holly A. Gurule

Teacher Education, Educational Leadership & Policy ETDs

Standardized tests are not new to the field of education. The problem with them seems to lie in the ways these tests are now being used as criteria for various high-stakes decisions such as teachers’ evaluations, school grades, school sanctions, and even graduation from high school with a diploma. With so many different uses for standardized tests, it is easy to see how our perception of them has become skewed.

This bounded case study of a small elementary school in a large urban district centered on shifting the focus away from looking at standardized tests as a negative, as mere …

Grades Versus Comments: Research On Student Feedback, Thomas R. Guskey

Educational, School, and Counseling Psychology Faculty Publications

Are comments on student work superior to grades? It depends.

Handwriting Performance Of Typical Second-Grade Students As Measured By The Evaluation Tool Of Children's Handwriting - Manuscript And Teacher Perceptions Of Legibility, Diane M. Long, James Conklin

The Open Journal of Occupational Therapy

Background: The purpose of the study was to describe scores achieved by typical second-grade students on the Evaluation Tool of Children’s Handwriting – Manuscript and to compare scores with teacher perceptions.

Method: As part of a larger study, the ETCH-M was administered to 74 second-grade students. Teachers scored classroom samples of handwriting assignments using a researcher-developed scale and scores were compared to ETCH-M scores to determine cutoff values for good versus poor handwriting.

Results: Mean scores for total word legibility, total letter legibility, and total numeral legibility were 88.82%, 84.30%, and 89.26%, respectively. Cutoff scores below 82% …

How Traditional Grading Contribute To Student Inequities And How To Fix It, Laura J. Link, Thomas R. Guskey

Educational, School, and Counseling Psychology Faculty Publications

Grades have long been identified by those in the measurement community as prime examples of unreliable measurement (Brookhart, 1994; Stiggins, Frisbie, & Griswold, 1989). What one teacher considers in calculating students’ grades may differ greatly from another teacher (Guskey & Link, 2019; McMillan, 2001; McMillan, Myran, & Workman, 2002). A major factor contributing to the unreliability of grades is teachers’ inclusion of aspects of students’ behavior in the grades they assign. Despite the recommendation of experts to separate behavior from academic achievement in formulating students’ grades, teachers at all grade levels typically include student behavior as a contributing factor in …

Improving Measurement And Expanding Meta-Analytic Knowledge: Social And Emotional Learning In Elementary And Early Childhood, Dana M. Murano

Dissertations, Theses, and Capstone Projects

In the last several decades, the development of student social and emotional skills in educational contexts has received much attention, both domestically and internationally. Whereas previous school-based educational practices had primarily focused on the teaching and testing of cognitive skills, we now recognize that there are constituents of academic success beyond the cognitive skills that are traditionally taught and tested, and the field of social and emotional learning (SEL) has emerged as a result. This two-study dissertation attempted to fill existing gaps in the development of SEL practices by exploring new horizons in both intervention and measurement with preschool and …

Measuring Media Literacy Inquiry In Higher Education: Innovation In Assessment, Evelien Schilder, Theresa Redmond

Journal of Media Literacy Education

The ability to critically access, analyze, evaluate, and create media messages is crucial in the process of becoming an informed and engaged citizen throughout life. Asking critical questions is not only a valuable dimension of media literacy, but also an indispensable aspect of participating in a democracy. Yet, measuring the effectiveness of media literacy is still a major challenge for the field. It is unclear to what extent people of all ages may engage in critical questioning habits with regards to media. To address this gap, we studied the changes in critical questioning habits for college-aged students enrolled in media …

Variations In Implementation Of Specifications Grading In Stem Courses, Mai Yin Tsoi, Mary E. Anzovino, Amy H. Lin Erickson, Edward R. Forringer, Emily Henary, Angela Lively, Michael S. Morton, Karen Perell-Gerson, Stan Perrine, Omar Villanueva, Marygeorge Whitney, Cynthia M. Woodbridge

Georgia Journal of Science

Specifications grading is an assessment strategy based on mastery learning, clear learning objectives, and frequent evaluations and feedback. Twelve instructors at a southeastern four-year public college implemented the specifications grading method across eight discrete courses in four STEM areas. In this modified assessment strategy, the students controlled their grades through multiple attempts, with limitations, on assessments of course objectives. The instructors designed and executed specifications grading in unique ways that aligned with their content areas, teaching beliefs, and individual teaching styles.

Preliminary observations suggest that, regardless of subject area, specifications grading can be used as an alternative to traditional assessment …

Context And Regulation Of Homeschooling: Issues, Evidence, And Assessment Practices, Janet F. Carlson

Buros Center: Professional Staff Publications

The article discusses salient factors that influence the current context within which homeschooling occurs. Individual states have applied various approaches to establish regulations that both preserve the rights of homeschooling parents and fulfill the state’s obligation to ensure that its residents receive the education to which they are constitutionally entitled. Case and ethnographic studies or research involving small and selected samples often appear in outlets associated with homeschool advocacy groups or in outlets that are not mainstream. The paucity of empirical evidence derived from methodologically strong research paradigms has led to little certainty about many aspects of homeschooling including its …

Main And Regional Campus Assessments Of Applicants To A Rural Physician Leadership Program: A Generalizability Analysis, Terry D. Stratton, Clarence Kreiter, Carol L. Elam

Behavioral Science Faculty Publications

While the selection of qualified applicants often relies, in part, on scores generated from a medical school pre-admission interview (MSPI), the growth of regional medical campuses (RMCs) – many with specialized rural tracks, programs, or missions – has challenged schools to accommodate a wider range of stakeholder input. This study examines the reliabilities of main (urban) and regional (rural) campus interviewers’ assessments of applicants to a Rural Physician Leadership Program (RPLP) located in the southeastern United States.

Data from RPLP applicants completing MSPIs on two campuses from 2009-2017 (n = 232) were examined in a generalizability analysis. In two separate …

Alignment Between Learning Objectives And Assessments In A Quantitative Literacy Course, Younggon Bae, Samuel L. Tunstall, Kathryn S. Knowles, Rebecca L. Matz

Numeracy

In this analysis, we examine how course assessment items were aligned with learning objectives in a quantitative literacy course at Michigan State University. The alignment analysis consisted of mapping assessment items to a list of operationalized learning objectives from the course. Our analysis shows how often the learning objectives are represented in assessment items, how often they are paired with other learning objectives, and how influential they are in contributing to a student’s course grade. In addition, through comparisons across four assessment types (e.g., exams and homework), we show how each learning objective was assessed differently within each assessment type. …

Comparison Of National Assessment Of Educational Progress (Naep) Reports And State Physical Education Mandates, Erick Alfonso

Seton Hall University Dissertations and Theses (ETDs)

There have been several studies that analyzed the relationship between physical education and academic achievement at a local level, but a longitudinal analysis across the United States has not been done. An understanding of the relationship between physical education policies across the United States and academic achievement at the state level may provide insight into best practices to ensure that comprehensive physical education policies are adopted in all states. Using data from the Society of Health and Physical Educators and The National Assessment of Educational Progress, I investigated the relationship between physical education policies and academic achievement. Minor relationships with …

A Closer Look At Toefl Speaking Assessment Test: A Guide To Teaching Toefl Ibt Speaking, Glen Ryan Alejandro

Master's Projects and Capstones

The lack of reliable speaking learning materials has been a problem of some non-native English speakers who are preparing for the TOEFL iBT Test (Test of English as a Foreign Language Internet-Based Test) and who are struggling most with the speaking section of said standardized test. Consequently, the lack of these reliable resources has been the main concern of TOEFL iBT speaking instructors whose main goal is to help meet the needs of these struggling students. As a result, many non-native English speakers who are preparing for the speaking section of the TOEFL iBT Test are not able to meet …

Sulitest®: A Mixed-Method, Pilot Study Of Assessment Impacts On Undergraduate Sustainability-Related Learning And Motivation, Alicia Mason

Faculty Submissions

A United Nations international collaboration between the Education for Sustainable Development (ESD) and the Principles for Responsible Management Education (PRME) resulted in the creation of Sulitest® (aka Sustainability Literacy Test) an open, online training and assessment tool freely available to higher education institutions globally. This study analyzes the effectiveness of the newly developed Sulitest® to not only measure sustainability literacy of higher education student populations, but also act as a catalyst for boosting affective learning outcomes by: (a) generating interest in sustainability-related issues, (b) improving sustainability-related understandings, and (c) enhancing students’ interests in the subject matter. In order to do …

Improving The Rigor Of Online Education: Exploring Characteristics Of Faculty And Syllabi Within An Online Program Assessment Process, Brad J. Hamel

All NMU Master's Theses

As online course enrollments grow, overall perception of rigor still lags compared to that of traditional face-to-face education. The purpose of this research was to tie faculty and syllabi characteristics to the rigor of online courses. This study explored the relationship between faculty and syllabi characteristics and performance on an online entry-level course design quality assurance assessment (pass, pass with concern, or fail). A decision tree analysis was used to predict the relationship of the independent (faculty and syllabi characteristics) and the dependent (entry-level course design assessment) variables. Findings suggest that faculty rank and writing intensive are key characteristics predictive …