Educational Assessment, Evaluation, and Research Commons^™

A Monte Carlo Study Of The Optimal Rank Order Relationship With Criterion Scores, Hongyan Cui

Dissertations

Appropriate scoring methods for tests should be based on theories of the construct domains of such tests (Messick, 1989). This is called structural fidelity (Loevinger, 1957), a necessary but not sufficient condition for construct validity (Keith & Kranzler, 1999). A situation reaction test (SRT) consists of items with alternatives to be ranked according to subjects’ best judgment. Traditional rank scoring methods assume unidimensionality, implying one scoring key for each item – a single external criterion. This is not appropriate when items elicit multidimensional responses determined by subjects’ best judgments based on possibly multiple internal criteria. The purpose of this study …

Assessment Matters #20, October 2014, Assessment In Action Committee

Assessment Matters @ WMU

No abstract provided.

A Rubric-Based Approach To Assessing Resources For Writing Student Learning Outcomes, Jesus Romero, Kyra Tilmon

Assessment Publications & Presentations

No abstract provided.

Capitalizing On Social And Transactional Learning To Challenge First-Grade Readers, Amanda Meyer, Roland K. Schendel

Reading Horizons: A Journal of Literacy and Language Arts

A classroom teacher capitalizes on social learning and reader response theories to challenge her accelerated first-grade readers by implementing literature circles. The aim of this action research was to identify a clear view of how to use literature circles with first-graders and what might be accomplished. Three constructs emerged from the interviews and observations that support the potential for using literature circles with primary students including: engagement and independence, reading benefits, and writing improvement. With respect to social learning and reader response theories, literature circles were found to be possible, practical, and beneficial for supporting the literacy perceptions and practices …

Career And Technical Education And The Malcolm Baldrige Quality Award In Education, Patricia Crum-Allen

Dissertations

This study examined Career and Technical Education (CTE) Centers in the State of Michigan and their potential alignment with the Malcolm Baldrige Quality Award in Education. CTE center leaders and their faculty were asked to provide their perceptions of how well their organizations meet Baldrige quality elements, using a version of the Baldrige Assessment Tool. The study further queried CTE center leaders and their faculty with regard to quality awards received and their desire to pursue an external quality award. Differences between CTE leaders and faculty responses were explored.

This study was quantitative in design using survey research to gather …

Teacher Trust Levels And How They Differ Between School Settings And Impact Teacher Involvement In Student Achievement Activities, Gretchen Kaeding Bergan

Dissertations

The No Child Left Behind Act of 2001 has brought accountability to the world of education. The Act’s purpose was to bring a standards-based educational reform process nationally by establishing measurable goals and high academic standards for students and teachers. The mandate to the states from the federal government specified that each state set up incremental assessments of all children, and schools have responded with significant efforts to improve student outcomes. One factor, teacher-to-teacher trust levels, has been shown through research (Bryk & Schneider, 1996) to improve professional working environments and student learning.

This was a comparison study of teacher-to-teacher …

Using Case Method To Explicitly Teach Formative Assessment In Preservice Teacher Science Education, Amy Elizabeth Bentz

Dissertations

The process of formative assessment improves student understanding; however, the topic of formative assessment in preservice education has been severely neglected. Since a major goal of teacher education is to create reflective teaching professionals, preservice teachers should be provided an opportunity to critically reflect on the use of formative assessment in the classroom. Case method is an instructional methodology that allows learners to engage in and reflect on real-world situations. Case based pedagogy can play an important role in enhancing preservice teachers’ ability to reflect on teaching and learning by encouraging alternative ways of thinking about assessment.

Although the literature …

Is There An Association Between Data-Informed District-Level Improvement Efforts And Data-Informed Instruction At The Classroom Level? An Empirical Examination Using National Schools And Staffing Survey (Sass) 2003-2004 Data, Xingyuan Gao, Jiangang Xia, Jianping Shen

Research and Creative Activities Poster Day

Introduction

What influences teacher’s instructional decision in the classroom? Our study uses nationally representative data to test whether there is an association between data-informed district/school-level improvement efforts and data-informed instruction at the classroom level.

A Comparison Of Students’ Perceptions Of Stress In Parallel Problem-Based And Lecture-Based Curricula, Sonia Wardley, Brooks Applegate, Deyab Almaleki, James Van Rhee

Research and Creative Activities Poster Day

Introduction

What is stress? Research asserts that stress is the mental state that results from an inability to cope (Burton 2004)

Why focus on stress?

• Persistent stress can lead to serious psychological problems such as interpersonal difficulties, depression, anxiety, and even suicide (Shapiro 2000)
• Several studies have found up to a third of medical students experience stress-related problems

The importance of this study comes from: A review of the extent literature suggests there is no systematic inquiry of the effects of stress experienced by students in LBL and PBL curricula in PA education

A Monte Carlo Study Of The Optimal Rank Order Relationships With Criterion Scores (Ororcs), Hongyan Cui, Brooks Applegate, Warren Lacefield, David Hartmann

Research and Creative Activities Poster Day

This dissertation study contributes to the field of measurement. It introduces a method to determine keys for a kind of test items that

1. Depict a specific situational context and provide a set of multidimensional alternative reactions to that situation for the subjects to rank according to their best judgment (SRT).
2. An examinee’s response to an item is a multidimensional ranking of the alternative options.
3. One administration of the SRT yields measures of multiple traits, so that there is no need to administer multiple criterion instruments.

Understanding Students' Ability To Communicate Mathematics In Developmental Mathematics Courses, Kirsty Eisenhart, Amy Galick

Assessment Grants

No abstract provided.

Examining And Testing The Use Of An Online Orientation, Betty D. Dennis

Assessment Grants

No abstract provided.

Assessment Matters #19, February 2014, Assessment In Action Committee

Assessment Matters @ WMU

No abstract provided.

Standardized Tests And Assessment In The Business School: A Pilot Study, Devrim Yaman

Assessment Grants

No abstract provided.

The Phoenix Project: An Intrusive Retention Initiative For At-Risk Students, Geralyn Heystek, Betsy Drummer, Chris Robinson

Assessment Grants

No abstract provided.

Goal Attainment Scaling: Measurement Of Student Learning And Teaching Clinic Outcomes: Program Evaluation In Progress, Ann M. Chapleau

Assessment Grants

No abstract provided.

Perceptions Of General Education Deans And Department Chairs Of Their Colleges As Learning Organizations, May Charmayne Mulligan

Dissertations

Higher education institutions are being pushed towards increased assessment practices because of rising costs, mass access, new delivery methods, and rising competition on the national and global scale. Academic deans and department heads are at the center of these assessment efforts, and assessment should guide changes within the organization for improvement. Learning organization theory provides a framework for creating an environment favorable to assessment and improvement. Six learning organization principles – learning, communication, measurement, problem-solving, structure, and vision – were used for this study.

An online questionnaire was used to capture the perceptions of deans and department heads as to …

Investigating Practices Of Research-Proven Multidimensional Teacher Evaluation Systems In Michigan Schools, George A. Aramath

Dissertations

Traditional teacher evaluation systems have little to no impact on changing teacher behavior or improving student achievement. Subsequently, studies and reports show the lack of and need for multidimensional teacher evaluation system. This deficiency and need is especially evident in Michigan due to the state's recent legislative mandate that their evaluation system must include multiple data as measures of educator effectiveness.

The purpose of this study is to investigate and describe how two Michigan public schools are attempting to build a multidimensional teacher evaluation system that includes the new Michigan mandate to incorporate evidence of student learning as a significant …

Improving The Design Of Cluster-Randomized Trials In Education: Informing The Selection Of Variance Design Parameter Values For Science Achievement Studies, Carl D. Westine

Dissertations

The purpose of this three-essay dissertation is to provide practical guidance to evaluators planning cluster-randomized trials (CRTs) of science achievement. In an educational setting, interventions are often administered at the cluster level, while outcomes are typically measured at the student level through standardized achievement testing. When evaluating an intervention, a CRT is appropriate because it allows for treatment to be modeled at a different level than the unit of analysis, and properly accounts for the violation of independence that occurs due to nesting. Accurately designing a CRT involves estimating variance parameters (i.e., intraclass correlations [ICCs] and percent of variance explained …

How Mature Is Your Assessment?, David S. Reinhold

Assessment in Action Conference

Included here is the PowerPoint presentation used during the plenary session at WMU Assessment in Action Day 2014. The session was an interactive session that allowed participants to determine how robust their assessment activities are in their units based upon answers to a series of questions developed by Karen Ann Tarnoff at East Tennessee State University. A short discussion period followed each question to allow attendees to comment on the questions. Cumulative results of the group were used to determine the maturity of assessment at WMU.

Assessment, Accreditation And Compliance: The Three Musketeers Or The Three Stooges?, Jody Brylinsky

Assessment in Action Conference

This presentation describes how assessment, accreditation and compliance fit together, in particular how they influence WMU’s strategic planning initiatives. While these three academic responsibilities work in concert to support the common goal of quality higher education, they often do so in some unusual ways? The purposeful connections and comedy of their relationship will be explored with illustrations from WMU pathway to reaccreditation in 2020.

Instituting A Proactive Risk Assessment Model: The Promise And Peril Of Predictive Approaches To Student Success, John Chew, Kevin Knutson, Edwin A. Martini

Assessment in Action Conference

In this session, the presenters, (all from the College of Arts and Sciences at WMU), describe a pilot project that attempts to predict the likelihood of student persistence based on a variety of factors, including non-cognitive assessment. Led by a graduate assistant who designed the survey tool and analyzed the data, they discussed the pros and cons of such an approach and the benefits of doing such work for the professional development of graduate students.

Western Michigan University Assessment In Action Day Conference 2014, Office Of Assessment And Undergraduate Studies

Assessment in Action Conference

No abstract provided.

Helpful And Reliable Assessment Resources, David S. Reinhold, Karen Stokes Chapo

Assessment in Action Conference

This document is the handout that was given to participants at the WMU Assessment in Action Day Conference 2014 who attended the concurrent session entitled, "Helpful and Reliable Assessment Resources". The presentation was given by David Reinhold, the Associate Provost for Assessment and Undergraduate Studies at Western Michigan University and Karen Stokes Chapo, the Assessment Programs Specialist at WMU. These are just a few of what Dave and Karen find to be helpful web resources on the topic of student learning outcomes assessment.

It's Hard To Hit A Moving Target: Establishing Clear Learning Objectives, Diane Rogers

Assessment in Action Conference

Are you getting the results that you want from your students? Do students know what you expect? Establishing and communicating clear learning outcomes from the beginning of a course can help to both improve the quality of student work and positively impact learning. This session will focus on writing clear learning and assessment expectations for academic courses and discuss strategies for implementing the outcomes in your course.

Beyond Satisfaction: Assessing (Learning And Other) Outcomes In Co-Curricular Areas, Kate Bates, Anne Lundquist, Cari Robertson, Ewa Urban, Chris Voss

Assessment in Action Conference

Most student support areas are fairly adept at designing surveys to capture satisfaction data. While important, there are other types of assessment that require our attention, not only so that we can impact student learning and improve programs and services for students and other constituents, but as an essential aspect of meeting revised Higher Learning Commission accreditation standards. This workshop will provide information about the various definitions and approaches to assessing student learning and operational outcomes in co-curricular areas and offer specific examples from four student affairs areas that may be applicable to other units on campus.

Sharing The Experience Of Nacada’S Assessment Of Academic Advising Institute 2014, Laura Ciccantell, Kevin Knutson, Lenore Yaeger

Assessment in Action Conference

Below is the description of the session that was presented by Laura Ciccantell, (College of Education Advising), Kevin Knutson, (College of Arts and Sciences Advising), and Lenore Yaeger, (College of Engineering and Applied Sciences Advising) at the WMU Assessment in Action Day Conference 2014. The attached document is a handout used to aid the participants during the presentation and as a tool to use for assessment after the session.

"Are you a student support unit that has a vision for teaching students? We argue that every office that interacts with students on our campus educates our students! So why not …

Beyond The Pencil: Expanding The Occupational Therapists’ Role In Helping Young Children To Develop Writing Skills, Hope K. Gerde, Tricia D. Foster, Lori E. Skibbe

The Open Journal of Occupational Therapy

Occupational therapists (OTs) play an important role in early childhood classrooms as vital members of the educational team, particularly for young children’s writing development. Children’s emergent writing is a foundational literacy skill, which begins to develop well before they enter elementary school. However, early childhood classrooms are lacking in supports for early writing development. OTs are experts in guiding the development of early writing skills in young children and, therefore, should be considered as critical members of the early literacy curriculum team. This paper identifies the critical role emergent writing plays in early childhood literacy development and how to effectively …

Collaboration Between Grant-Making Organizations, Evaluators, And Program Staff Related To Standardized Evaluation Requirements, Kristin M. Everett

Dissertations

Nonprofit organizations have received funding from grant-making organizations and foundations for many years. These funds have been used to support programs developed by the nonprofit organization. Over the years, two critical developments have occurred between grant-making organizations and nonprofit organizations related to evaluation and collaboration. First, grant-making organizations and nonprofit organizations are combining funds and resources to collaborate on projects. Second, program evaluation has become an essential piece of many grant-funded projects.

Historically, nonprofit organizations receiving funding from multiple sources were required to implement multiple monitoring and evaluation plans and write a report for each funder. These reports use staff …

Measuring Performance Excellence: Key Performance Indicators For Institutions Accepted Into The Academic Quality Improvement Program (Aqip), Paul J. Ballard

Dissertations

Given growing interest in accountability and outcomes, the North Central Association's Higher Learning Commission developed a new path for accreditation, the Academic Quality Improvement Program (AQIP). The goal is to infuse continuous improvement and quality in the culture of higher education, and to blend traditional accreditation with the philosophy and techniques of the Baldrige Performance Excellence Program. Yet, little is known about the key performance indicators (KPI) that colleges and universities are using to improve their quality and continuous improvement processes.

To this end, my research involved a content analysis of the Systems Portfolios submitted for accreditation by 34 purposefully …