Educational Assessment, Evaluation, and Research Commons^™

Continuous Improvement In Student Affairs, Jen Wells

Assessment in Action Conference

Imagine booking a trip but never getting on the plane, or buying a movie ticket but never walking into the theater. Or what if you fully planned a program, but you never executed it? Barring environmental or extraneous factors, it’s hard to imagine not following through in these scenarios. Now, think about learning valuable information through an assessment project, but never using it. This should be equally unfathomable. The data collected in our assessments should be meaningful by informing our work and helping us continually improve. This presentation offers strategies on how to make data-informed decisions, demonstrate why this is …

Writing & Connecting Learning Outcomes Across The Co-Curricular Environment, Marilyn Duke, Ciji Heiser, Anthony Helms

Assessment in Action Conference

This session focuses on the two primary areas of writing learning outcomes and then connecting outcomes to a co-curricular map. In the first part of the session, participants will work through a learning outcomes model to draft new outcomes or revisit existing outcomes and receive feedback from peers and session facilitators. In the second part of the workshop, the discussion will shift to how to connect learning outcomes across multiple programs and services to develop co-curricular maps. Outcome maps help staff identify areas of strength in student learning outcomes and gaps in outcome development. Participants will have the opportunity to …

Graduate Student Success: Developing A Holistic Assessment Process For The Electronic Portfolio Using The Acpa/Naspa Professional Competency Areas, Ramona B. Lewis, Casey Dornhecker

Assessment Grants

The purpose of this project is to develop a holistic and sustainable assessment process for the Performance Driven Leadership electronic portfolio that is completed as part of the M.A. in Educational Leadership Higher Education and Student Affairs Leadership (HESA) concentration.

Assessing Creative Growth In The Frostic School Of Art, William Charland

Assessment Grants

This project examined the suitability of the Torrance Test of Creativity as a direct measure of determining growth in student creativity from matriculation to graduation among students enrolled in B.F.A. and B.A. programs across the areas of the Frostic School of Art (FSoA). Test takers were anonymous, representing an opportunity sample of incoming and outgoing student cohorts. This research project’s aim was to determine if creativity, as distinct from techniques and skills, increased between the first year and graduation among students in the FSoA.

2019 Western Michigan University Assessment In Action Conference, Office Of Assessment And Undergraduate Studies

Assessment in Action Conference

Included here is the schedule and session descriptions from the 10th Annual WMU Assessment in Action Conference held on March 15, 2019. The conference, sponsored by the Office of Assessment and Undergraduate Studies at Western Michigan University, is a one-day event where those new to outcomes assessment can come and learn fundamental knowledge about the field, but it is also a place where those who have worked as leaders in outcomes assessment can come and extend their knowledge and skills and network with colleagues. Conference offerings included learning opportunities on various assessment topics for both academic and learner supports units.

Collaborative Learning For High Voltage Engineering Education, Pablo Gomez, Edmund Tsang

Assessment Grants

The goal of this project is to evaluate the applicability and efficacy of a collaborative learning environment for high voltage (HV) engineering education. This environment is based on the interaction of two pedagogical approaches: computer simulation and experimental/laboratory work. Two courses on HV engineering were offered in parallel in the Fall 2018 semester – one taught at Western Michigan University focusing on computer simulation and the other one taught at Mississippi State University focusing on experimental/laboratory work.

A Rather Different ‘Operating System’ For Courses, Learning And Assessments — A Formative, Objectives-Mastery, Modular System (Fomms), David Schuster

Assessment Grants

This project involves a different way of running courses and assessments, in line with a very formative philosophy on learning. Being a whole system, it involves several components working together. Topics are designed as a sequence of small natural ‘learning units’, each with specific learning objectives. The aim is for all students to eventually attain high-level ‘mastery’ of all topic objectives, given the opportunity. Structured ‘learning tasks’ occur during learning, and ‘mastery assessments’ after topic completion. Both are formative rather than summative, in that students have an opportunity to learn from feedback and try again for mastery (assessment for learning). …

Cliftonstrengths For Students: Developing Engaged And Thriving Students Through Strengths Awareness And Development, Adrienne Fraaza, Ewa Urban

Assessment Grants

The Office of Student Transitions embarked upon its second pilot year of implementing Gallup's CliftonStrengths for Students into First-Year Seminar (FYS) courses. Strengths development focuses on improving self-awareness and well-being, giving students a tool to understand how to leverage their natural talents for success. Nearly half of the FYS instructors chose to integrate Strengths activities and development into their courses. Final data indicated an increase in self-awareness, confidence, and affirmation of major/career choices. Nearly 89% of pilot students said that Strengths should be implemented in all FYS courses.

Assessing The Assessors: Potential Barriers To Engaged Writing Assignments, Lisa Singleterry, Susan L. Caulfield

Assessment Grants

The aim of this project is to inform departments and schools within the College of Health and Human Services of how their faculty, in general, approach writing-related requirements with their courses. Authors adapted 3 surveys to measure faculty perceptions about writing, including faculty approaches to assigning writing, faculty comfort levels with writing and how equipped they are to assess writing. Data will be entered into SPSS for data analysis that will be used to inform needs for department and faculty support or policy development.

Continuous Improvement In Academic Affairs, Jen Wells

Assessment in Action Conference

As faculty, we are asked to focus on teaching, research, and service; how we manage our work is often pre-determined by how we allocate time to these three elements. With that, the question is where assessment and continuous improvement fit. This presentation will offer a model of continuous improvement and ideas on how it can be integrated in your current work and not as a check the box exercise. We will also review how assessment can be scaffolded and implemented at the graduate level.

Curriculum Mapping As A Path For Success, Susan L. Caulfield, David Paul, Cynthia Visscher

Assessment in Action Conference

This workshop engages participants in a conversation about their knowledge and experience with curriculum mapping. Examples of curriculum mapping are provided, along with assessment outcomes related to such mapping. Participants will be asked to articulate a class or program-level outcome and workshop facilitators will be able to assist with the definition and creation of outcomes. Such outcomes will then be related back to curriculum mapping. Participants will leave with two different examples of curriculum mapping, as well as some experience in engaging the process of curriculum mapping.

What's Your Assessment - Why?, Jen Wells

Assessment in Action Conference

Accountability. Accreditation. Resources. Student Success. Student Learning. Student Development. External Factors. Program Review. All of these terms and more are associated with assessment. In this keynote, I want to go back to the why! Why is this important? Why should we do this? What is the why of assessment and how do our own experiences translate and transfer into the concept of continuous improvement.

Marginal Commentary: Are Students And Instructors On The Same Page?, Maria Ornella Treglia

Teaching/Writing: The Journal of Writing Teacher Education

This article reports the findings of 141 student questionnaires and interviews with six teachers to investigate whether first-year students’ preferences align with their teachers’ written commentary in composition classes in an urban community college. Results show that students appreciate and rely on teacher commentary and prefer it to be clear, detailed, and supportive. They indicated that commentary that combines the message with a positive phrase works best. Teachers, on the other hand, were not aware of their students’ needs and preferences, and expressed self-doubt and frustration about their students’ reception of written commentary.

Journal Of Communication Pedagogy, Complete Volume, 2019

Journal of Communication Pedagogy

No abstract provided.

Reducing Secondary Trauma And Compassion Fatigue In The Dark Side Of Interpersonal Communication Classrooms, Mary E. King, Albra Wheeler

Journal of Communication Pedagogy

This essay describes the experiences faculty may encounter when teaching tough topics. When professionals are in the position as the individual who cares for, hears about, or witnesses the trauma and suffering of others, they might themselves be at risk for experiencing vicarious stress, or secondary trauma. If ongoing and untreated, this traumatic stress can morph into compassion fatigue, which can impede professional success and contribute to burnout. This essay reflects on the experiences of teaching the Dark Side of Interpersonal Communication where students openly shared course-related personal experiences. We reflect on how to successfully manage the delicate climate of …

Best Practices For Teaching Discussion As Part Of High School Common Core State Standards, Mitchel Stengel, Leah Nolan, David Donnick, Wesley Skym, Anna M. Wright

Journal of Communication Pedagogy

Instructional discussion is a teaching method used in many classrooms across grade levels. In fact, the Common Core State Standards promote the use of instructional discussion in secondary classrooms (Common Core State Standards Initiative, 2018a). Students, however, are not always taught best practices for engaging in a discussion and may feel unprepared to participate. As a result, discussions may not produce the dynamic learning opportunity they are intended to foster. This essay provides 10 tips for high school teachers to prepare students in the high school classroom to engage in a meaningful classroom discussion effectively in order to ensure students …

Developing A Digital Voice: Embedding Digital Communication Platforms, Networks, And Technologies In The 21st-Century Classroom, James M. Proszek

Journal of Communication Pedagogy

Introductory communication courses are an invaluable resource for improving students’ abilities to speak confidently, passionately, and persuasively, while also inspiring them to become more engaged citizens. However, digital media present new opportunities and challenges for designing courses that are relevant to students’ personal and professional interests and goals. Instructors who incorporate digital communication platforms, networks, and technologies into their classrooms can better prepare students to meet the complex demands of the technologically-mediated 21st century. In this essay, I offer 10 best practices for developing students’ digital media literacies within multiple communication contexts.

Implication Of Item And Test Differential Functioning In Criterion-Referenced Tests, Xiaojie Hu

Dissertations

In a compensatory criterion referenced test (CRT) it is common that the test developer maximizes the test information at or near the cut score location. This is believed to produce the highest classification accuracy. However, a recent study showed that the optimal location to maximize test information for highest classification accuracy (CA) and consistency (CC) also depends on factors such as test length, cut score location, the examinee ability distribution, and the test model applied for latent trait estimation (Wyse & Babcock, 2016). Moreover, when unintentional secondary dimensions are also measured (nuisance dimension caused by test construction, test standardization, or …

Limited By Design: Expanding The Boundaries Of Schooling Policies From Demanded Passivity To Transformative Practice, Natalia T. Carvalho-Pinto

Masters Theses

Schooling is often imagined as the great equalizer and the primary venue towards personal and collective liberation. However, hegemonic ideals and practices have limited the ability for educators to craft and utilize policy in a way that best benefits their students. In this thesis, I utilize the state of Michigan’s Common Core curriculum to examine the limits beyond policy and practice. Specifically, I argue that the demands for student passivity and institutional measurable as signs of success, prohibit student growth. Throughout the document, I describe hope in the ability to instill a variety of transformative educational theories and practices that …

The Effect Of Item Stem And Response Option Length On The Item Analysis Outcomes Of A Career And Technical Education Multiple Choice Assessment, Tina M. Koepf

Dissertations

Critical decisions in Career and Technical Education (CTE) can be based on assessment outcomes, requiring an essential focus on continuous improvement initiatives to provide increased multiple choice (MC) assessment validity. Previous research has determined that the presence of flawed test items negatively impacts student success; therefore, an MC taxonomy has been established and used in the assessment industry. While rigorous and successful Career and Technical Education (CTE) programs that enroll high-level academic students exist, many CTE programs enroll students with a wide spectrum of academic ability. Because many CTE students often have inferior reading skills, it is necessary to take …

Online Evaluation Training Program, Lori Wingate

Academic Leadership Academy

The Western Michigan University (WMU) Evaluation Center and Extended University Programs are developing an online professional development program in evaluation. The learner-centered program will include a mix of live and self-paced instruction, including webinars, videos, text, assessments, and practice exercises. The following qualities set this program apart from existing academic and ad hoc evaluation training programs:

Equity-informed: As a critical aspect of high-quality evaluation practice, the curriculum is infused with guidance on how to address equity throughout the evaluation process.

Practical: Instruction is connected to practical application so learners can use what they learn right away.

Accessible: Barriers to learning …

Apology Not Accepted: The Impact Of Executive Rhetoric, Communication Strategies, Media Coverage And Time On Crisis Management And Public Perception During Major Oil Spills, Todd D. Harcek

Dissertations

This research builds upon the scholarship in crisis communication strategies and apologia employed by companies during a crisis. Within the expansive domain of crisis management, this work evaluates the media coverage and crisis communication strategies during three major oil spills: the Exxon Valdez, the American Trader, and the BP Gulf oil spill. The purpose of this study is to provide practitioners, researchers and others additional insight into the impact of media coverage on prior, current and future crises and how this coverage, coupled with the communication strategies of companies during a crisis, can influence public awareness or perception. Furthermore, …

Empirical Estimates Of Design Parameters Used In A Priori Power Analyses For Planning School-Randomized Studies In Select Asian Countries, Ran Shi

Dissertations

A cluster-randomized trial (CRT) is one type of randomized controlled trial (RCT) that is often more feasible for evaluating the effectiveness of an educational intervention due to the multi-level structure of the data (e.g., students nested in schools). In order for a CRT to generate rigorous evidence, it is essential that is it adequately powered to detect an effect of a given size. As such, it is common for researchers to conduct a priori power analyses to determine how large a sample the CRTs will require to detect an effect of a given magnitude. Previous work has shown that two …

University Leaders’ Experiences In Advancing The Development And Transformation Of Emerging Saudi Public Universities, Abdulrahman Hamed Alsulami

Dissertations

For several decades, Saudi higher education comprised seven to eight public universities concentrated in the main cities of the country. Since the beginning of the new millennium, the Kingdom of Saudi Arabia has heavily invested in the higher education sector. Thus, it witnessed a remarkable rate of expansion from 8 to 28 higher education institutions during a relatively short period between 2003 and 2013. This renaissance in the sector involved several different processes, including transforming, merging, restructuring, and establishing new colleges and/or programs, administrative posts, and key components for inclusive and modern campuses.

From a contextual perspective, there is a …

Response Shift Bias: An Examination Of Measurement Invariance In Self-Reported Change, Katherine Nelson Daniels

Dissertations

Traditional pre-test (TpT)/post-test (PT) and retrospective pre-test (RpT)/post-test (PT) designs are used to collect data on self-reported measures to assess the magnitude of change that occurs from interventions. If measurement invariance does not exist across the measurement occasions within these research designs, it is inappropriate to compare mean group differences that result from the intervention and derive inferences about change. The theory of response-shift suggests a subject’s understanding of a subject matter at TpT may not be the same as their understanding of a subject matter at PT, and that as a result, the construct measured at these measurement occasions …

Measuring Intercultural Competency Of Wmu Study Abroad Students, Lee Penyak, Ryan Rounds

Assessment Grants

Using the Global Perspective Inventory (GPI), this project employs pre- and post-tests to measure the extent that study abroad programs influence the development of intercultural competency, if and how students move from monocultural to multicultural frames of mind, and the impact that international travel experiences have on fortifying the University’s third pillar of global engagement. This information will 1) be shared with current faculty directors to build intercultural learning objectives into program curriculum, 2) help WMU SA revise initial and final study abroad proposals by prospective faculty directors, and 3) be used to restructure pre-departure orientations provided by WMU SA …

Using Elearning For Assessment, Katrina Davidson, Alyssa Moon

Assessment in Action Conference

There are many ways to formatively and summatively assess learner knowledge using an online learning environment. This session will give an overview of these tools and examine use cases.

Assessing In-Course Projects And Students' Interest In Science, Heather Kasper, Mary Anne Sydlik, Susan R. Stapleton

Assessment Grants

An important question in higher education is "How do we best support students so that they stay at Western Michigan and remain or become interested in careers in one of the sciences?” One way seems to be getting students interested in research, which can spark and maintain their continued interest in STEM fields. Traditional laboratory activities often involve repeating classical experiments to reproduce known results rather than engaging students in experiments with the possibility of true discovery. Instead, we need to actively engage students to interest them in STEM and to keep them in the field. Our project uses data …

"What Is My Target?": Clarifying Student Learning Outcomes, Anthony Helms, David Paul

Assessment in Action Conference

This session will provide an overview of best practices in writing feasible and measurable learning and process outcomes, including hands-on time to work on your own outcomes.

Why Are We Doing This?, Linda Suskie

Assessment in Action Conference

Many find assessment to be a pointless paper-pushing exercise. Done right, however, it is essential to helping students learn and succeed. This presentation will summarize research on strategies that help students learn and succeed and explain the role of assessment in those strategies.