Educational Assessment, Evaluation, and Research Commons^™

An Exploratory Study On The Teaching Of Evidence-Based Decision Making, Erica Baarends, Marcel Van Der Klink, Aliki Thomas

The Open Journal of Occupational Therapy

Background: There is no clear guideline on how to teach students evidence-based decision making (EBDM), so this study aimed to assess the impact of an educational intervention on students’ EBDM skills.

Methods: This was an explorative mixed-method study of 12 undergraduate occupational therapy students and their teacher. The teaching was aimed at increasing self-efficacy and cognitive skills in EBDM. Semi-structured interviews were conducted to gather the students’ perceived learning benefits. Before and after the intervention, a self-efficacy questionnaire, a critical thinking test, and scored generic cognitive skills in an argument were used as measures of learning achievements. Content …

Modelling Student Misconceptions Using Nested Logit Item Response Models, Mustafa Yildiz

Dissertations

Student misconceptions have been studied for decades from a curricular/instructional perspective and from the assessment/test level perspective. Numerous misconception assessment tools have been developed in order to measure students’ misconceptions relative to the correct content. Often, these tools are used to make a variety of educational decisions including students’ achievement level, instructional method effectiveness, and curriculum related achievement progress. These tools have included qualitative and quantitative assessment methods.

The quantitative analysis of misconceptions has mostly relied on classical test theory methods of test construction related to total raw score, percentage of correct responses, and/or percentage of misconception responses. More recently, …

The Quick Analysis For Principals In The Alignment Of Administrative Plans Within The School Improvement Framework At The Middle School Level, James Vanden Heuvel

Masters Theses

High performing schools require administrators to align the continuous improvement efforts with the professional development activities while evaluating teaching and learning for fidelity. The administrative plans of school improvement, professional development, and teacher evaluation systems systemically aligned begin with an administrator who can see the important interaction of these three plans. The increased pressure for improved achievement from stakeholders, coupled with competing demands on time for the school administrator, leads towards a need for efficient tools for improvement. School administrators are creating administrative plans that are in isolation or disconnected from each other which leads to a lack of cohesion. …

Using Concept Maps To Monitor Knowledge Structure Changes In A Science Classroom, Leah J. Cook

Dissertations

The aim of this research is to determine what differences may exist in students’ structural knowledge while using a variety of concept mapping assessments. A concept map can be used as an assessment which connects concepts in a knowledge domain. A single assessment may not be powerful enough to establish how students’ new knowledge relates to prior knowledge. More research is needed to establish how various aspects of the concept mapping task influence the output of map creation by students. Using multiple concept maps and pre-instruction and postinstruction VNOS instruments during a 16-week semester, this study was designed to investigate …

A Meta-Analysis Of Energy Savings From Lighting Programs In Michigan, Teryila Ephraim Amough

Dissertations

In order to fill the gap in aggregates savings, the challenges faced by electric utilities for the demand of lighting energy are addressed by lighting efficiency programs. The shrinking capacity and electrical grid reliability call for improved ways to evaluate energy saving programs with evaluation methods that are robust in determining the impact of lighting programs. This study employed meta-analysis as an evaluation method to determine energy savings, impact, emissions of greenhouse gases (GHG), pollutants, and health effects from lighting programs in Michigan. The findings of the study showed the programs impact in Hedges’ g of 0.36 for the overall …

Do Incentives Increase Response Rates To An Internet Survey Of American Evaluation Association Members?, Lyssa N. Wilson

Research and Creative Activities Poster Day

This randomized experiment examines the effect of different incentive types on online survey response rates. The primary objective of this study was to determine the most effective incentive type to increase response rates to online surveys of American Evaluation Association (AEA) members, specifically.

Assessing Inter-Professional Teamwork Through Low-Fidelity Simulation, Lisa Singleterry, Sally Vliem, Kristine Gibson, Scott Gaynor

Assessment Grants

In 2003, it was determined that health-related education needed to shift from discipline-specific silos and into the arena of inter-professional teams. Given this goal, teamwork – which includes communication and the opportunity to learn about, with, and from multiple health disciplines – is the focus of this assessment grant. A sub-group of the faculty in training team of WMU professionals accelerating clinical education redesign (PACER) chose to utilize a low-fidelity simulation (one which uses verbal descriptions, case studies, discussion and role play to explore a particular topic, engaging all simulation participants in an active learning atmosphere) for medical students, clinical …

Pre-Service Teachers Establishing Professional Identities In The 21st Century, Hsiao-Chin Kuo, Susan Piazza

Assessment Grants

This project investigates how pre-service teachers document their developing professional identities and learning outcomes through an ePortfolio assignment. The goal is to strengthen the program’s ability to improve and document student learning outcomes. Direct measures that determine the development of effective teachers is becoming more challenging with traditional paper and print-based artifacts. An increasing number of programs in higher education utilize web-based portfolios to measure learning outcomes. The ePortfolio developed in an undergraduate literacy course will make use of formative and summative assessments. Data is collected via multiple sources, including pre-service teachers’ ePortfolios and other class artifacts, classroom observations and …

Honing In On What To Measure, Kristen Mckinney

Assessment in Action Conference

Institutional and departmental outcomes are framed in necessarily broad terms as we articulate the key values and goals of our work. Bridging the gap between those broad outcomes and more tailored and measurable program and service specific outcomes can be difficult. This session will present some ideas and strategies for developing outcomes and guide participants through the process of drafting outcomes for their own programs.

Assessment As Ally: Focusing On What Matters, Gloria Rogers

Assessment in Action Conference

Too many times, the assessment of student learning becomes an exercise of futility where faculty time is squandered and the result is many data and no information that is useful for the improvement of student learning. When properly understood and implemented, the assessment process can provide meaningful results that inform faculty about the achievement of outcomes, the alignment of the curriculum, and what can be done to improve student learning. Topics which will be discussed are the importance of well‐defined outcomes, curriculum mapping, the development of systematic processes, and reporting results for action.

Bba Quantitative Skills: Assessing The Prerequisites, Matthew Ross, A. Michelle Wright

Assessment Grants

We examine determinants of prerequisite quantitative skill among Bachelor of Business Administration (BBA) students. Preliminary results suggest that undergraduate students are deficient in quantitative skills, even among a sample of finance students whom show greater quantitative ability than the mean BBA student. Mean math quiz performance among 164 student participants substantially lags faculty established expectations. We extend beyond traditional BBA pedagogy research by incorporating the Attitudes Toward Mathematics Inventory (ATMI) and language analysis in a business finance context. We find strong evidence for learning theory explanations involving quantitative prerequisite ability and non-cognitive measures. However, theories involving knowledge decay and demographic …

Topic Generation And Development In The Freshman Research Paper, Edward J. Eckel

Assessment Grants

This study will help better understand the cognitive process students go through in developing their research paper topics by examining how students in selected sections of a freshman engineering writing class come up with an initial topic for their course research paper, how they refine or develop that topic into something they can find adequate research on, and how they shape that topic into a purpose statement for their final research paper. Research methodologies include short online surveys, in-person interviews at the end of the semester, and use of a rubric to assess the final submitted purpose statements for their …

Examining Changes In Students' Intercultural Development In Student Affairs Professional Preparation, D. Eric Archer

Assessment Grants

This project is to assess if the multicultural mission of the WMU Higher Education and Student Affairs masters’ program (i.e., “recognize the centrality of diversity and inclusiveness to higher education and to effectively and ethically lead and serve in a dynamic educational culture”) is being fulfilled. Students in the HESA program will complete the Intercultural Development Inventory assessment during their first semester in the program (Fall 2016) and again in their final semester (Spring 2018). This pre-and post-test assessment will provide information regarding students’ intercultural development as a result of completing the program, which places an emphasis within the curriculum …

Western Michigan University 2017 Assessment Conference, Office Of Assessment And Undergraduate Studies

Assessment in Action Conference

Included here is the schedule and session descriptions from the 8th Annual WMU Assessment Conference held on March 17, 2017. The conference, sponsored by the Office of Assessment and Undergraduate Studies at Western Michigan University, is a one-day event where those new to outcomes assessment can come and learn fundamental knowledge about the field, but it is also a place where those who have worked as leaders in outcomes assessment can come and extend their knowledge and skills and network with colleagues. Conference offerings included learning opportunities on various assessment topics for both academic and student supports units.

Reconnecting With “Why” To Move Beyond Assessment Sticking Points, Kristen Mckinney

Assessment in Action Conference

We have made great steps forward in developing assessment infrastructure on our campuses, but often find ourselves still struggling with really making use of the data we collect. Assessment can feel like a chore rather than something that truly has meaning for our work. In this session, Dr. McKinney will discuss how re‐connecting assessment with our core goals and purposes can help refocus our energy and attention in ways that help us use data more effectively to tell our story.

Using Data To Tell Your Story, Kristen Mckinney

Assessment in Action Conference

We are increasingly faced with a dizzying array of options for how to display data. How do we make informed choices about what will really aid the user in telling a compelling story versus those that are just “pretty”? This session will review aspects of visual perception that influence how we process visual information, as well as discuss the messages conveyed by various visual displays and how to select options that best aid the reader in understanding key data points.

Defining Outcomes, Gloria Rogers

Assessment in Action Conference

This workshop will engage participants in the writing of measurable outcomes and how they relate to scoring rubrics. Group discussion and formal feedback will enhance the understanding of the critical role of well‐defined outcomes in the assessment process.

Transforming Data Into Information, Gloria Rogers

Assessment in Action Conference

Data are collected but how do we transform the data to meaningful information that can be used to improve student learning. A case study will be used to explore the criteria for an assessment plan that will lead to actionable information.

Are The Leadership Behaviors Of K–12 Leaders In Mid-Western Urban School Districts Influenced By Their Beliefs And Attitudes Regarding Spirituality?, Ericka M. Taylor

Dissertations

Working in a stressful and chaotic environment can be difficult for building principals in urban school districts (DeNisco, 2013; Stokley, 2002; Thompson, 2004). Presumably, the job is only expected to get more arduous with the increasing demands to improve student achievement (Allison, 2012; Kimball & Sirotnik, 2000; Queen & Schumacher, 2006). What is quite apparent today is that the role and responsibilities of the principal have changed dramatically over the past twenty years (Carlson, 2012; Hill-Yeverton, 2003) shifting from a factory model of management of learning to one in which teachers and principals have to create more student-centered and project-based …

Expanding A Path Analytic Model Of Quality Management To Include The Management Of Safety, Bruce J. Ladewski

Dissertations

The Occupational Health and Safety Administration (OSHA) reports that every year 4.1 million American workers are seriously injured from workplace accidents at an annual direct and indirect cost estimated at hundreds of billions (OSHA, 2012). At the public level, actions to reduce the workplace deaths, injuries, and associated costs in the form legislation. At the private level, actions taken include implementing behavior based safety programs focused on various ways of encouraging employee behaviors to reduce reported injuries. The most recent move to drive down worker injuries focuses on the implementation of management systems by adopting principles used to prevent product …

From Descendants Of Refugees To First-Generation College Students: The Untold Story Of Southeast Asian American College Students’ Lived Experience, Vunsin Hiew Doubblestein

Dissertations

Aggregate data on Asian Pacific Americans (APAs) may show APAs to have exceptionally high educational attainment and economic mobility compared to other racial groups (Museus & Buenavista, 2016). In reality, various APA sub-groups face significant obstacles to academic access and success and are found at both the highest and lowest levels of the achievement, as well as educational attainment and income spectrums. In spite of the growing awareness of and scholarship on the complexity of APA student experience in higher education, there is limited research on Southeast Asian Americans (SEAAs) who have come to the U.S. as refugees, the circumstances …

Investigating Personality Traits Of Registered Nurses: Implications For Personnel Evaluation, Stephanie N. Means

Dissertations

The United States is currently facing a registered nurse (RN) shortage that is expected to reach the millions in the next few years. Since the beginning of the century, healthcare professionals and administrators have strived to develop and implement ways to increase recruitment and retention in the nursing field. It has been recognized that the key to recruitment and retention is to get the right nurse in the right place, which means focusing on clinical proficiencies as well as personality qualities and characteristics. Personality assessment has long been used to evaluate person-job (P-J) fit of applicants for positions across the …

Killing Me Softly: The Missed Education Of African American Females, Margaret Daphne Cattenhead

Masters Theses

Using a literature review and personal experience, this paper aims to examine the paradoxes and barriers of African American girls in education. The educational system has left black females without support to ensure consistent results across a variety of settings and environments. There are a variety of voices on what barriers African American girls face and how those barriers influence their educational progress. Despite some progress, many African American girls are at risk for suspension, expulsion, and grade retention. They are at risk to be pressured into dominant culture norms for behavior, to be regarded as subordinate, to be negatively …

Awareness And Use Of Electronic Health Records In Entry-Level Occupational Therapy And Occupational Therapy Assistant Curricula, Louis F. Dmytryk, Tina M. Deangelis

The Open Journal of Occupational Therapy

The Accreditation Council for Occupational Therapy Education (ACOTE) requires programs to instruct entry-level occupational therapy (OT) and occupational therapy assistant (OTA) students in technology that may include electronic documentation systems, distance communication, virtual environments, and telehealth (standard B1.8). At this time, there are no publications describing if and how electronic health record (EHR) instruction is implemented in entry-level OT and OTA programs. The purpose of this study is to investigate awareness and use of EHRs in entry-level OT and OTA curricula. Respondents from 76 nationally accredited entry-level programs (two OT doctoral, 24 OT masters, two OT combined bachelors/masters, and 48 …

Designing Impact Studies In Education In The Past 15 Years: Have We Made Any Progress?, Jessaca Spybrook

Spring Convocation

Dr. Spybrook presented this at the 2017 Spring Convocation as the Emerging SCholar for 2016 at Western Michigan University. It was based on research done with Ran Shi and Ben Kelcey.

Superintendents’ Response To The Revised School Code Requiring Merit Pay In Selected Michigan School Districts, Mark A. Tompkins

Dissertations

In 2010, Public Act 205 was passed and signed into law, revising Michigan School Code section 1250. This revision required compensation for teachers to be based on job performance and job accomplishments. Compensation based on performance or merit is a significant departure from existing practice wherein teachers are paid a step scale salary based on years of service and educational qualifications. The purpose of this phenomenological study was to investigate the lived experience of nine superintendents as it relates to the implementation of section 1250 of PA 205 in their respective districts. Participants were divided into three categories based on …

A Phenomenological Study Of Practicing Educators’ Personal And Collaborative Experiences Within A Climate Of High Stakes Individual Accountability, Kathy L. Stewart

Dissertations

With the national focus in education turning to increasing student achievement and closing achievement gaps between demographic groups, federal and state policy has extended responsibility and high stakes accountability for student growth and achievement. Overall, student achievement status and elimination of achievement gaps between demographically defined sub-groups of students remain cornerstones of accountability under ESEA and now ESSA. Under the Obama Administration, however, accountability for academic growth was extended to individual classroom teachers and school and district-level administrators through federal policy provisions tying student growth to performance evaluation. As a result, many states, including Michigan, enacted legislation to mandate high …

Shaping Leadership: Leadership Standards In The Organizational Language Of University Programs, R. Ben Meador Iv

Dissertations

According to institutional theory, organizations in complex fields like education seek legitimacy by aligning formal descriptions of their activities to institutional rules supported by regulators, professional groups and exemplar organizations. This study presents a new technique for measuring isomorphism and the strength of institutional forces within fields using quantitative content analysis of organizational language. This method is deployed to measure alignment between the formal descriptions of 321 university-based programs for educational leaders and themes from the Professional Standards for Educational Leaders (i.e., technology, finance, research, and learning) and the influence of state licensure policy and professional affiliation in the University …

A Formative Evaluation Of A Program Providing Support To College Students With Autism Spectrum Disorder, Kourtney Kay Bakalyar

Dissertations

Young adults with autism spectrum disorder (ASD) are attending institutions of higher education more than ever before (Smith, 2007). All college students with disabilities have the right to accommodations under the American with Disabilities Act Amendments Act of 2008 (ADAAA; P.L. 110-325); however, these accommodations frequently do not address barriers that prevent students with ASD from successfully completing their academic programs. The Autism Services Center at a public, four-year university provides services to its college students with ASD to address difficulties with communication and social skills, unique emotional and behavior characteristics, insufficient executive function, and difficulties with independent living skills. …

Three Empirical Investigations Into The Logic Of Evaluation And Valuing Practices, Satoshi Ozeki

Dissertations

This three-paper dissertation investigates the logic of evaluation proposed by Michael Scriven using two different methods of inquiry: content analysis and a survey method. Scriven’s logic of evaluation is one of the central concepts in evaluation, but has not been empirically investigated. Therefore, the purpose of this three-paper dissertation is to accumulate empirical knowledge on Scriven’s logic of evaluation to investigate the extent to which evaluators follow his logic and their perceptions of and familiarity with his logic. Furthermore, this dissertation aims to explore valuing practices in terms of Scriven’s logic of evaluation. Scriven’s logic of evaluation is important because …