Educational Assessment, Evaluation, and Research Commons^™

Student Perspectives On Grading In Higher Education Stem Courses, Christopher Sean Burns, Daniela D.S. Digiacomo, Jennifer Pusateri, Katherine Ritchie, Christine Wusylko

Information Science Faculty Publications

Purpose

Despite a growing awareness of instructional practices that support student learning and well-being, higher education classrooms tend to still be dominated by the use of rigid approaches to assessment and course design. This study aims to understand the experiences of students in courses that implement a universal design for learning (UDL) pedagogical approach and an ungrading assessment approach. Design/methodology/approach

Leveraging a qualitative methodological approach, authors interviewed 13 graduate students in two library and information science technology courses and 11 undergraduate students in a computer science course in discrete mathematics about their experiences with ungrading within the context of UDL-designed …

Leadership And Literacy Assessment Systems: A Multi-Case Study Of System Coherence And Assessment Use In Mississippi Elementary Schools, Kristin P. Walters

Theses and Dissertations--Educational Leadership Studies

This qualitative embedded multiple-case study examined how school-level literacy leadership shaped the coherence and use of literacy assessment systems in five Mississippi Elementary Schools. Situated within the context of Mississippi’s recent literacy gains and the policy environment emerging from the Literacy-Based Promotion Act, this study explored how literacy leaders guide assessment systems to support coherence between literacy instruction and assessment practice.

Data were collected through semi-structured interviews with 25 educators across five elementary schools in different districts. Analysis was conducted using descriptive and values coding organized into an analysis matrix that facilitated within-case and cross-case exploration of patterns in distributed …

Effects Of Constant Time Delay Using Multiple Acquisition Exemplars On The Generalization Of Self-Instruction To Complete Vocational Tasks For High School Students With Intellectual Disability, Anna Norby

Theses and Dissertations--Early Childhood, Special Education, and Counselor Education

This study presents findings from a multiple probe design that evaluated the effects of constant time delay with multiple acquisition exemplars on the acquisition and generalization of self-instruction to complete vocational tasks for high school students with intellectual disability. The research included three high school students with intellectual disability, and data were analyzed across baseline, intervention, and generalization conditions. Instruction targeted the use of self-instruction procedures embedded within vocational tasks, with multiple exemplars incorporated during acquisition to promote stimulus generalization. Participants were screened using a set of pre-arranged vocational tasks to identify existing skill levels and determine appropriate instructional targets, …

Enhancing International School Implementation Capacity: Supporting Contextualized Learning At An International Baccalaureate School In China, Christopher J. Hayden Iv

Theses and Dissertations--Educational Leadership Studies

International schools allocate substantial resources to implement progressive curricular innovations; however, the advantages offered by these developments are contingent on their effective execution within classroom settings. This mixed methods action research (MMAR) study examined the implementation of the International Baccalaureate (IB) Global Context (GC) Strategy in the Middle Years Programme (MYP) at an international school in China and asked how the school might strengthen its capacity to enact this and similar innovations. Following a sequential MMAR design, a reconnaissance phase combined a whole-staff survey (n=87) with qualitative interviews to diagnose the individual, organizational, and contextual factors shaping implementation. Analyses identified …

Between The Rails: Experiential Education And Its Impact On Compliance And Consistency In Racetrack Surface Management, Heather Mackenzie

Theses and Dissertations--Community & Leadership Development

Recent regulatory requirements within the Thoroughbred racing industry have increased expectations for consistent collection and reporting of racetrack surface data. Despite clear standards, implementation in daily operations remains inconsistent, highlighting a gap between regulation, knowledge, and practice. This study examined the impact of an experiential education curriculum on racetrack personnel’s self-efficacy and reporting consistency.

Using a quantitative design, data were collected through pre- and post-surveys measuring self-efficacy and records of the number of days surface data were entered. Descriptive statistics, paired-samples t-tests, and effect size analyses were used. Results showed a statistically significant increase in self-efficacy (p < .001) with large effect sizes, indicating meaningful gains in confidence and comfortability. Reporting consistency did not show a statistically significant change (p = .885).

Overall, experiential education …

Academic Departments As Microfoundations Of Institutional Governance, Karley A. Riffe, Meghan J. Pifer

Educational Policy Studies and Evaluation Faculty Publications

Considerations of governance in US higher education do not typically include the academic department. This results in a knowledge gap about the role of these central organizing units of academic work in the governance of colleges and universities. Through the concept of microfoundations, we explore concrete ways in which departmental governance matters, how it may be challenging, and strategies for fostering effective governance within academic departments.

Exploring Pathways To Persistence And Self-Efficacy Among Historically Underrepresented Community College Students: A Mixed Methods Action Research Study, Sharmy Rene Davis

Theses and Dissertations--Educational Leadership Studies

Research indicates that postsecondary institutions have struggled to retain historically underrepresented college students. These students continue to experience lower levels of persistence and college self-efficacy compared to their peers (Monjaras-Gaytan & Sánchez, 2023).

This dissertation is a mixed methods action research study that identified the academic challenges historically underrepresented college students face. This study focused on diagnosing the problem, engaging in multiple Reconnaissance phases, and using those findings to plan a contextually responsive intervention.

Findings indicate that historically underrepresented college students may benefit from enhanced academic support, stronger advising relationships, and greater opportunities to build peer connections. The findings of …

Critical Discourse Analysis Of Nil Policies: Examining Institutional Control And Economic Opportunities For Division I Football Athletes In Power Four Universities, Allen T. Alvarez

Theses and Dissertations--Educational Policy Studies and Evaluation

The adoption of Name, Image, and Likeness (NIL) policies has redefined the economic landscape of collegiate athletics, expanding opportunities for student-athletes and reshaping institutional authority. This dissertation nvestigates how NIL policy language at Division I universities frames economic possibilities for football student-athletes and sustains systems of governance. Through a Critical Discourse Analysis of policy documents from Power Four and mid-major institutions, this study reveals that NIL participation is persistently conditioned by institutional approval, compliance monitoring, and ambiguous definitions, especially regarding boosters and collectives.

Policy language presents NIL as an opportunity but embeds institutional control through oversight structures. The analysis further …

The Operationalization Of Student Success: How Institutions And Divisions Of Student Affairs Define And Align Non-Academic Student Success With External Accountability Expectations, Madeline N. Trudeau

Theses and Dissertations--Educational Policy Studies and Evaluation

This qualitative multi-case study investigates how higher education institutions and their divisions of student affairs publicly discuss and define non-academic student success through their strategic planning, within the context of external pressures from state legislatures and accrediting bodies. Grounded in Organizational Learning Theory, Transition Theory, and Theory of Student Involvement and Engagement, this study analyzes the alignment and divergence between external policy expectations and internal institutional and divisional strategic plans.

The findings reveal a fundamental tension; while external stakeholders increasingly demand quantifiable outcomes tied to economic productivity - like post-graduate wages and employability metrics - divisions of student affairs continue …

Student Familiarity With The Periodic Table Of The Elements: Results From Cued-Recall And Eye-Tracking Assessments On Memory, Victor A. Okuo

Theses and Dissertations--Chemistry

Learning element symbol–name associations and the spatial organization of elements on the periodic table is a foundational step in learning chemistry, supporting later understanding of chemical formulas, equations, bonding, and stoichiometry. Although students often rely on memorization strategies to learn periodic table content, this task is challenging due to the large number of elements and the apparent ambiguity in matching some element symbols to their names.

This study explores students’ recall of element names when given element symbols as cues and their knowledge of element locations on the periodic table when given element names as cues. The study also examines …

Successfully “Reframing” Assessment Work Due To Macro And Micro Pressures, Julene L. Jones, Mary K. Thompson, Beth Janetski, Sarah Wu

Library Presentations

The Grand Challenges Project supports global collaborations informing equitable and evidence-informed practices for higher education assessment practitioners. The Grand Challenge’s Equity Action Makers Team conducted twenty interviews in 2023/2024 with assessment professionals across higher education. Several participants explained that they have adjusted their approach to equity work to successfully complete their projects or achieve their goals. They did this in response to various macro and micro influences, such as recognizing their own positionality, meeting grant funding requirements, dealing with institutional financial or cultural pressures, navigating political pressures outside their influence, or fostering supportive networks of practitioners.

Pushing Boulders: Exploring The Barriers To Conducting Equity-Informed Assessment, Beth Janetski, Sarah Wu, Julene L. Jones, Mary K. Thompson

Library Presentations

In the quest for equitable student success, assessment professionals from higher education and supporting organizations face a myriad of challenges. As part of the Grand Challenges in Assessment Project, twenty-four leaders in equity-informed assessment practices were interviewed to share their experiences. They revealed the internal and external barriers they navigate daily, from mental stress and isolation to structural obstacles. These professionals are on the front lines, striving to ensure all students have equal access and outcomes, despite the uphill battle against entrenched resistance and apathy. Their stories highlight the resilience and dedication required to push for meaningful change.

Evaluating Experiential Learning In Undergraduate Agricultural Education: A Content Analysis Of Course Syllabi From A College Of Agriculture, Tyler Newberry

Theses and Dissertations--Community & Leadership Development

This study evaluated the presence and depth of experiential learning within undergraduate course syllabi from the Martin-Gatton College of Agriculture, Food, and Environment. Through a content analysis of 41 syllabi across five academic majors, the study examined how course objectives and assignments aligned with Kolb’s Experiential learning cycle (1984), Webb’s depth of knowledge (1997), and Joplin’s Mini to maxi Scale (1981). A structured coding instrument was used to quantify the frequency of learning elements across these three frameworks. The findings revealed that Abstract Conceptualization and Reflective Observation were the most common stages in both objectives and assignments, while Concrete Experience …

Assessing Generalization Along Side Acquisition Using System Of Least Prompts Versus Video Modeling Plus System Of Least Prompts Of Simulated Vocational Tasks, Victoria B. Shafer

Theses and Dissertations--Early Childhood, Special Education, and Counselor Education

The purpose of this study was to assess if system of least prompts alone or video modeling plus system of least prompts was more effective when teaching a student with a moderate intellectual disability three simulated vocational tasks using paper materials. Additionally, generalization was assessed using real-world materials. An adapted alternating treatment design was used to compare conditions. The results indicated that system of least prompts alone was more effective than video modeling plus system of least prompts. There was no demonstration of generalization.

Just What Is Equity And What Is It Doing In A Nice Field Like Assessment, Julene Jones, Glenn Phillips, Ciji Heiser

Library Presentations

This presentation outlines a literature review on equity and assessment, drawing from the article Making a Grand Contribution: Professional Development Resources for Leveraging Assessment Data to Advance Equity. The review will explore how assessment practices can promote equity, identifying professional development strategies that support equitable data use in higher education. The findings will inform institutional leaders, faculty, and assessment professionals on embedding equity within assessment processes. This session will engage participants in discussions about the history of equity in assessment and its current uses, equipping them with tools to advance equitable assessment practices and institutional change.

From Data To Action: Leading Assessment In Polarizing Times, Julene L. Jones, Ciji Heiser, Renée Delgado-Riley, Heather Strine-Patterson, Kevin Wright

Library Presentations

In an era of increasing political and social polarization, assessment professionals lead within complex landscapes to ensure data is used to drive meaningful change. This session features a panel of professionals exploring how to move from data collection to organizational change while leading assessment efforts with integrity and impact. Participants will answer a series of questions focused on current and evolving challenges, examples of their practice, and strategies they use to center student and organizational success in the assessment journey.

Rasch Model Utilization Towards Developing An Instrument For Measuring Creative Drive, David W. Stenulson

Theses and Dissertations--Educational Policy Studies and Evaluation

Creativity is commonly assessed across undergraduate programs as a graduation requirement. However, a considerable amount of literature suggests that creativity is a complex, multidimensional construct, and more efficient ways of assessing it should be explored with multiple measures representing different creative aspects. This exploratory pilot study sought to take steps in developing a supplemental instrument designed to measure creative drive, a theoretical construct with predictive potential for future creative output. In order to gain a greater understanding of the nature of creative drive and gauge accuracy of initial thinking in terms of item writing, the Creative Drive Assessment (CDA), a …

Investigating The Fits And Functionality Of The Community College Survey Of Student Engagement Using Rasch Modeling, Olukemi O. Kolawole

Theses and Dissertations--Educational Policy Studies and Evaluation

Student engagement is a central variable in higher education research as it influences academic achievement, retention, and the development of problem-solving skills. The Community College Survey of Student Engagement (CCSSE) is a vital tool for assessing student engagement in community colleges. Previous validation studies were mainly conducted in the mid-to-late 2000s and have used Classical Test Theory frameworks despite the known limitations of this approach. The present study seeks to evaluate the psychometric properties of the CCSSE survey using a Rasch Analysis framework. The two CCSSE datasets were obtained from the Center for Community College Student Engagement at The University …

From Engagement To Clarity: Understanding Career Exploration Activities And Their Influence On Career Clarity Among High School Students, Odunayo O. Omotilewa

Theses and Dissertations--Education Sciences

This study examined the associations between career exploration activities and career clarity among high school students in the U.S.A. Using data from the 2022 Program for International Student Assessment, collected from a national sample of 15 to 16-year-olds across 154 schools, the researcher first identified student and school factors associated with engaging in career exploration. The study then examined the influence of these activities on career clarity. Specifically, the analysis focused on internships, job shadowing, job fairs, career counseling, completing career questionnaires, researching careers and colleges online, attending college tours, and financing information, and on the level of engagement in …

Investigating The Psychometric Properties Of A Set Of Local Stem Literacy Instruments In Rasch Model, Lijun Shen

Theses and Dissertations--Educational Policy Studies and Evaluation

This study investigates the psychometric features of the See Blue STEM literacy inventories (SBSTEMs, including the elementary version SB-E and the middle school version SB-M) to determine their validity in measuring STEM literacy. Using data from elementary and middle school students participating in the See Blue STEM camp in 2019 and 2021-22, this research employed the Rasch Partial Credit Model (PCM) to analyze item-level performance and identify DIF items across various demographic subgroups (race, gender, free-lunch status, and potential first-generation status) and camp delivery modes (online versus in-person). Results indicated that both the elementary (SB-E) and middle school (SB-M) versions …

Examining The Psychometric Properties Of The Classroom Whiteness Scale (Claws) Through Rasch Partial Credit Analysis, Jaylene T. Patterson

Theses and Dissertations--Educational, School, and Counseling Psychology

This pilot study examines how whiteness manifests in K-12 classrooms by developing and evaluating the Classroom Whiteness Scale (ClaWS), an instrument designed to assess in-service teachers’ salience and engagement with whiteness-informed practices. ClaWS consists of 21 items across two subscales: one measuring the salience of whiteness-related beliefs in teachers’ pedagogical decisions and the other capturing the frequency of these practices in classroom settings. Rooted in critical whiteness studies, the scale operationalizes key dimensions of whiteness, including ethnocentric monoculturalism, white standardization, white silence, right to comfort, white saviorism, and colorblind racial ideology.

Using Rasch Partial Credit Model (RPCM) analysis, this study …

Open Alex, Kyle Demes

Research Analytics Summit 2024

Research organizations of all shapes and sizes need information on the research activities at their institutions and other institutions around the world to effectively plan, implement, and evaluate their research strategies. Historically, that information was only accessible through closed commercial licenses. This session will introduce OpenAlex, the most comprehensive and only completely open index of the world’s research ecosystem. The presentation will first provide an overview of OpenAlex, highlighting the importance of openness and transparency in research analytics, and will walk-through how to perform common research analyses in OpenAlex.

You Are What You Eat ... And Enter Into Data Fields, Hansa Magee, Nick Bilyk

Research Analytics Summit 2024

No abstract provided.

Finding And Hiring A Research Analyst Unicorn, Hansa Magee, Nick Bilyk

Research Analytics Summit 2024

As more data analytics programs pop up, we see graduates with these degrees and certificates in our applicant pool. However, the analyst that works in higher ed administration is not like a data scientist that looks for biomarkers in genomic data or the programmer that writes in R all day. What are the skills needed for this job? What exactly is this job that we are hiring for? Should managers prioritize applicants with analytics certificates? Or is experience with sponsored projects more important? We (your presenters) don’t have perfect answers but we (the collective conference attendees) can come up with …

Building Rural Resilience: A Rural And Farm Youth Safety And Well-Being Curriculum, Kelly Mcfarland

Theses and Dissertations--Community & Leadership Development

Mental health is a growing concern in today’s youth (CDC, 2023). The suicide rates in youth have been on a rise, increasing the need for mental health literacy (CDC, 2023). Mental health is often neglected in people’s lives, especially in rural areas (Boyd et al., 2008). This study answers four research questions: 1) Has the Building Rural Resilience: A Rural and Farm Youth Safety and Well-being Curriculum increased students' knowledge to recognize stress factors among rural communities and farmers? 2) Has the Building Rural Resilience: A Rural and Farm Youth Safety and Well-being Curriculum increased students' knowledge to recognize stress …

Using Agency Data For Research Development, Lou Hurst

Research Analytics Summit 2024

This session will explore ways to locate, analyze, and present sponsor-provided data to guide research development. Individual investigators, who may be proposing to a funder for the first time, often want to know about funding success rates, as well as more nuanced insight into the kinds of projects agencies tend to favor. They may also benefit from insight into changing workforce demographics and career trajectories in their discipline. At the departmental and institutional level, leadership seeks information about trends in funding availability and mechanisms, and in patterns of grants made, to better position themselves for these opportunities. As a research …

The Balancing Act: Data Analytics And Radical Invention, Andrea Christelle, Baron Wolf

Research Analytics Summit 2024

Metrics and analytics are becoming indispensable tools for guiding decision-making and resource allocation. That’s why we are all at this conference. However, what happens when the path to innovation veers off the beaten track, to such an extent that metrics don’t capture it? This session will delve into the delicate balance between relying on metrics and fostering revolutionary invention by exploring how research administrators can strategically harness analytics while also nurturing the creative space necessary for groundbreaking discovery.

This thought-provoking session will engage participants in a dialog that considers the unanticipated nature of invention, and how it can be fostered, …

From Policy Sources To Research Information Management Sources: Incorporating Multiple Bibliographic Data Sources Into Dashboard Analytics, Rachel Miles

Research Analytics Summit 2024

As the availability of bibliographic data becomes more diverse and unique, so do the complexities of those sources. As a result, it can be challenging and daunting to analyze such data, especially from a single research analytics service request. In this session, the presenter will demonstrate examples of building dashboards to analyze data for research outputs of a college using data from a research information management system, a policy database, and a bibliographic database. Attendees will learn about the advantages of using RIM systems to perform analytics as well as their shortcomings; they’ll also learn how to export data from …

Making The Most Of What You Have: Data To Optimize Space And Achieve Research Goals, Paul Leef

Research Analytics Summit 2024

Every institution has a strategic goal for more research funding and results. The drive for funding is inextricably tied to competition for top researchers, and competition for researchers is directly related to investments in and optimization of research space on university campuses. The case for focusing on space is critical when you expect each lab to produce $1 million or more in expenditures every 4 years. How can you use data analytics to optimize space allocation for research? How can you creatively use under-utilized spaces for short- and long-term research investments? This session will discuss Carnegie R2 and R1 and …

Introducing A ‘Research Impact Roadmap’ In A Young University: Challenges And Opportunities, Roopa Rajashekar

Research Analytics Summit 2024

‘Research Impact’ as a defined concept is expanding and garnering more attention globally. What was once positioned more narrowly and often confused with basic outcomes reporting has since risen to a primary focus for sponsor agencies across all stages of R&D. Hence, building a comprehensive research impact framework provides a critical map for universities to strengthen the R&D ecosystems in which it engages in innumerable ways.

Khalifa University of Science and Technology (KU), located in Abu Dhabi, UAE, is a relatively a young university, by Times Higher Education definition). Even so, KU ranked 238 in the QS World University Rankings …