Educational Assessment, Evaluation, and Research Commons^™

Student Perspectives On Grading In Higher Education Stem Courses, Christopher Sean Burns, Daniela D.S. Digiacomo, Jennifer Pusateri, Katherine Ritchie, Christine Wusylko

Information Science Faculty Publications

Purpose

Despite a growing awareness of instructional practices that support student learning and well-being, higher education classrooms tend to still be dominated by the use of rigid approaches to assessment and course design. This study aims to understand the experiences of students in courses that implement a universal design for learning (UDL) pedagogical approach and an ungrading assessment approach. Design/methodology/approach

Leveraging a qualitative methodological approach, authors interviewed 13 graduate students in two library and information science technology courses and 11 undergraduate students in a computer science course in discrete mathematics about their experiences with ungrading within the context of UDL-designed …

Leadership And Literacy Assessment Systems: A Multi-Case Study Of System Coherence And Assessment Use In Mississippi Elementary Schools, Kristin P. Walters

Theses and Dissertations--Educational Leadership Studies

This qualitative embedded multiple-case study examined how school-level literacy leadership shaped the coherence and use of literacy assessment systems in five Mississippi Elementary Schools. Situated within the context of Mississippi’s recent literacy gains and the policy environment emerging from the Literacy-Based Promotion Act, this study explored how literacy leaders guide assessment systems to support coherence between literacy instruction and assessment practice.

Data were collected through semi-structured interviews with 25 educators across five elementary schools in different districts. Analysis was conducted using descriptive and values coding organized into an analysis matrix that facilitated within-case and cross-case exploration of patterns in distributed …

Effects Of Constant Time Delay Using Multiple Acquisition Exemplars On The Generalization Of Self-Instruction To Complete Vocational Tasks For High School Students With Intellectual Disability, Anna Norby

Theses and Dissertations--Early Childhood, Special Education, and Counselor Education

This study presents findings from a multiple probe design that evaluated the effects of constant time delay with multiple acquisition exemplars on the acquisition and generalization of self-instruction to complete vocational tasks for high school students with intellectual disability. The research included three high school students with intellectual disability, and data were analyzed across baseline, intervention, and generalization conditions. Instruction targeted the use of self-instruction procedures embedded within vocational tasks, with multiple exemplars incorporated during acquisition to promote stimulus generalization. Participants were screened using a set of pre-arranged vocational tasks to identify existing skill levels and determine appropriate instructional targets, …

Enhancing International School Implementation Capacity: Supporting Contextualized Learning At An International Baccalaureate School In China, Christopher J. Hayden Iv

Theses and Dissertations--Educational Leadership Studies

International schools allocate substantial resources to implement progressive curricular innovations; however, the advantages offered by these developments are contingent on their effective execution within classroom settings. This mixed methods action research (MMAR) study examined the implementation of the International Baccalaureate (IB) Global Context (GC) Strategy in the Middle Years Programme (MYP) at an international school in China and asked how the school might strengthen its capacity to enact this and similar innovations. Following a sequential MMAR design, a reconnaissance phase combined a whole-staff survey (n=87) with qualitative interviews to diagnose the individual, organizational, and contextual factors shaping implementation. Analyses identified …

Between The Rails: Experiential Education And Its Impact On Compliance And Consistency In Racetrack Surface Management, Heather Mackenzie

Theses and Dissertations--Community & Leadership Development

Recent regulatory requirements within the Thoroughbred racing industry have increased expectations for consistent collection and reporting of racetrack surface data. Despite clear standards, implementation in daily operations remains inconsistent, highlighting a gap between regulation, knowledge, and practice. This study examined the impact of an experiential education curriculum on racetrack personnel’s self-efficacy and reporting consistency.

Using a quantitative design, data were collected through pre- and post-surveys measuring self-efficacy and records of the number of days surface data were entered. Descriptive statistics, paired-samples t-tests, and effect size analyses were used. Results showed a statistically significant increase in self-efficacy (p < .001) with large effect sizes, indicating meaningful gains in confidence and comfortability. Reporting consistency did not show a statistically significant change (p = .885).

Overall, experiential education …

Academic Departments As Microfoundations Of Institutional Governance, Karley A. Riffe, Meghan J. Pifer

Educational Policy Studies and Evaluation Faculty Publications

Considerations of governance in US higher education do not typically include the academic department. This results in a knowledge gap about the role of these central organizing units of academic work in the governance of colleges and universities. Through the concept of microfoundations, we explore concrete ways in which departmental governance matters, how it may be challenging, and strategies for fostering effective governance within academic departments.

Exploring Pathways To Persistence And Self-Efficacy Among Historically Underrepresented Community College Students: A Mixed Methods Action Research Study, Sharmy Rene Davis

Theses and Dissertations--Educational Leadership Studies

Research indicates that postsecondary institutions have struggled to retain historically underrepresented college students. These students continue to experience lower levels of persistence and college self-efficacy compared to their peers (Monjaras-Gaytan & Sánchez, 2023).

This dissertation is a mixed methods action research study that identified the academic challenges historically underrepresented college students face. This study focused on diagnosing the problem, engaging in multiple Reconnaissance phases, and using those findings to plan a contextually responsive intervention.

Findings indicate that historically underrepresented college students may benefit from enhanced academic support, stronger advising relationships, and greater opportunities to build peer connections. The findings of …

Evaluating Experiential Learning In Undergraduate Agricultural Education: A Content Analysis Of Course Syllabi From A College Of Agriculture, Tyler Newberry

Theses and Dissertations--Community & Leadership Development

This study evaluated the presence and depth of experiential learning within undergraduate course syllabi from the Martin-Gatton College of Agriculture, Food, and Environment. Through a content analysis of 41 syllabi across five academic majors, the study examined how course objectives and assignments aligned with Kolb’s Experiential learning cycle (1984), Webb’s depth of knowledge (1997), and Joplin’s Mini to maxi Scale (1981). A structured coding instrument was used to quantify the frequency of learning elements across these three frameworks. The findings revealed that Abstract Conceptualization and Reflective Observation were the most common stages in both objectives and assignments, while Concrete Experience …

Assessing Generalization Along Side Acquisition Using System Of Least Prompts Versus Video Modeling Plus System Of Least Prompts Of Simulated Vocational Tasks, Victoria B. Shafer

Theses and Dissertations--Early Childhood, Special Education, and Counselor Education

The purpose of this study was to assess if system of least prompts alone or video modeling plus system of least prompts was more effective when teaching a student with a moderate intellectual disability three simulated vocational tasks using paper materials. Additionally, generalization was assessed using real-world materials. An adapted alternating treatment design was used to compare conditions. The results indicated that system of least prompts alone was more effective than video modeling plus system of least prompts. There was no demonstration of generalization.

Rasch Model Utilization Towards Developing An Instrument For Measuring Creative Drive, David W. Stenulson

Theses and Dissertations--Educational Policy Studies and Evaluation

Creativity is commonly assessed across undergraduate programs as a graduation requirement. However, a considerable amount of literature suggests that creativity is a complex, multidimensional construct, and more efficient ways of assessing it should be explored with multiple measures representing different creative aspects. This exploratory pilot study sought to take steps in developing a supplemental instrument designed to measure creative drive, a theoretical construct with predictive potential for future creative output. In order to gain a greater understanding of the nature of creative drive and gauge accuracy of initial thinking in terms of item writing, the Creative Drive Assessment (CDA), a …

Investigating The Fits And Functionality Of The Community College Survey Of Student Engagement Using Rasch Modeling, Olukemi O. Kolawole

Theses and Dissertations--Educational Policy Studies and Evaluation

Student engagement is a central variable in higher education research as it influences academic achievement, retention, and the development of problem-solving skills. The Community College Survey of Student Engagement (CCSSE) is a vital tool for assessing student engagement in community colleges. Previous validation studies were mainly conducted in the mid-to-late 2000s and have used Classical Test Theory frameworks despite the known limitations of this approach. The present study seeks to evaluate the psychometric properties of the CCSSE survey using a Rasch Analysis framework. The two CCSSE datasets were obtained from the Center for Community College Student Engagement at The University …

Investigating The Psychometric Properties Of A Set Of Local Stem Literacy Instruments In Rasch Model, Lijun Shen

Theses and Dissertations--Educational Policy Studies and Evaluation

This study investigates the psychometric features of the See Blue STEM literacy inventories (SBSTEMs, including the elementary version SB-E and the middle school version SB-M) to determine their validity in measuring STEM literacy. Using data from elementary and middle school students participating in the See Blue STEM camp in 2019 and 2021-22, this research employed the Rasch Partial Credit Model (PCM) to analyze item-level performance and identify DIF items across various demographic subgroups (race, gender, free-lunch status, and potential first-generation status) and camp delivery modes (online versus in-person). Results indicated that both the elementary (SB-E) and middle school (SB-M) versions …

Making The Most Of What You Have: Data To Optimize Space And Achieve Research Goals, Paul Leef

Research Analytics Summit 2024

Every institution has a strategic goal for more research funding and results. The drive for funding is inextricably tied to competition for top researchers, and competition for researchers is directly related to investments in and optimization of research space on university campuses. The case for focusing on space is critical when you expect each lab to produce $1 million or more in expenditures every 4 years. How can you use data analytics to optimize space allocation for research? How can you creatively use under-utilized spaces for short- and long-term research investments? This session will discuss Carnegie R2 and R1 and …

Introducing A ‘Research Impact Roadmap’ In A Young University: Challenges And Opportunities, Roopa Rajashekar

Research Analytics Summit 2024

‘Research Impact’ as a defined concept is expanding and garnering more attention globally. What was once positioned more narrowly and often confused with basic outcomes reporting has since risen to a primary focus for sponsor agencies across all stages of R&D. Hence, building a comprehensive research impact framework provides a critical map for universities to strengthen the R&D ecosystems in which it engages in innumerable ways.

Khalifa University of Science and Technology (KU), located in Abu Dhabi, UAE, is a relatively a young university, by Times Higher Education definition). Even so, KU ranked 238 in the QS World University Rankings …

Unlocking Opportunities: Leveraging Generative Ai In University-Sponsored Programs Operations, Lisa A. Wilson, Tubal Yisrael, Alex Wagner

Research Analytics Summit 2024

Explore the transformative potential of generative AI in university-research administration. As universities strive to enhance research capabilities and support a culture of innovation, the need for efficient and effective management of sponsored programs has become paramount. This presentation will share lessons from deploying a sponsored programs’ Large Language Model Chatbot and how it optimizes research administration operations and unlocks opportunities. By harnessing the power of GenAI, a university office of sponsored programs chatbot can develop training materials, policies and SOPs. It can offer immediate support and guidance by analyzing queries and providing real-time responses, empowering staff members to overcome challenges …

The Co-Curriculum Of College Student Employment: Perspectives From Supervising Practitioners, Brittany Begley Wildman

Theses and Dissertations--Educational Policy Studies and Evaluation

College student employees are not only serving the overall mission of their institutions, but also developing career readiness skills through daily interactions while at work. In this study, I explore the perceptions of the supervisors of student employees as both managers and student affairs professionals. Through one-on-one interviews with supervisors, discussing their interactions with students, I explore the implicit and explicit curricula they provide to their students using a curriculum analysis framework. I then compare the resulting learning outcomes against those identified in industry standards. The supervisors described supporting student employees' personal development during the student employment experience as well …

Manrrs Advisors And Their Perceptions Of Their Mentor Strategies, Dallas L. Cooks

Theses and Dissertations--Community & Leadership Development

Minorities in Agriculture Natural Resources and Related Sciences, also known as MANRRS, is an organization with chapters nationwide. Their main focus is to provide professional development, academic advancement, and networking to those in agriculture and related sciences. This research focuses on the advisors of MANRRS chapters across the country and at various institutions. Advisors were asked a series of questions through a Qualtrics survey about their mentor strategies with their students. Chapter Advisors were asked demographic, Likert Scale, and questions based on their personal MANRRS experience. As a result of the data, five research questions were answered. Chapter Advisors and …

De Aquí, Y De Allá: Education, Culture, And Identity Development Of Transnational Latinas, Elizabeth Harris-Medina

Theses and Dissertations--Educational Policy Studies and Evaluation

De Aquí, y De Allá: Education, Culture, and Identity Development of Transnational Latinas This ethnographic case study examines the experiences of three transnational Latina women, descendants of families from Degollado, Jalisco, Mexico, exploring how their migration history and educational backgrounds influence their life trajectories. Focusing on these three women, the study aims to better understand their challenges and inform the development of resources to support their success in the U.S. education system. Using ethnographic interviews and Ecological Systems Theory, the research highlights how places and social systems shape views and life paths, offering insights into how best to support these …

A Multilevel Nonlinear Approach Of Piecewise Regression For Detecting Turning Points: Developing An Alternative Platform With An Application Of Timss, Jing Zhang

Theses and Dissertations--Education Sciences

Studies in recent years have explored the methods of identifying the turning points when dealing with the non-linear relationship. Four approaches applied by researchers, the eyeball approach, the establishment approach, the theory-driven approach, and the data-driven approach, are all under the traditional piecewise regression framework when seeking turning points. Thus, the purpose of this study is to introduce a multilevel piecewise regression model to identify the turning point beyond the traditional piecewise regression, as a completely non-linear approach.

Data used for this study is TIMSS 2019 United States sample. TIMSS is a project guided by the International Association for the …

Survey Development And Validation: Student Self Expectations Of The First-Year College Experience Survey (The Se-Fye Scale), Min Xiao

Theses and Dissertations--Education Sciences

First year at a university is often a transitional stage for students where they have opportunities to grow and develop in various areas and get familiar with a collegiate environment. Understanding what first-year students expect to navigate and succeed in their first year at the university is crucial. Their expectations reflect how well they prepared for university, affecting their behaviors and eventually influencing their intentions to continue higher education. One of the primary purposes of this study was to design a survey to measure student self-expectations of the first-year college experience (the SE-FYE scale) by applying a structured survey development …

Evaluating The Differential Effects Of Using A Technology-Based Antecedent Intervention In Comparison To A Technology-Based Treatment Package, Rachel Fosnaught

Theses and Dissertations--Early Childhood, Special Education, and Counselor Education

This study investigated the differential effects of using non-contingent reinforcement in the form of attention as an antecedent intervention in comparison to using non-contingent reinforcement in addition to self-monitoring as a treatment package. A multi treatment single case research design was utilized in this study. Levels of student’s on-task behavior were measured to evaluate the differential effects of an antecedent intervention in comparison to the combination of an antecedent and consequent intervention. This study also only used treatments that were technologically delivered to the participants.

Minimum High School Graduation Requirements And Teacher Compensation, Allison Johnson

Theses and Dissertations--Educational Policy Studies and Evaluation

In the last forty years of educational reform, significant changes have been made to the number and type of high school credits students are required to complete for graduation. As these requirements have changed, the demand for higher-level academic courses has increased while teacher shortages, particularly in high-demand areas such as math and science, have continued to increase. This study uses data from the National Center for Education Statistics (NCES) Student and Staffing Survey to examine the relationship between minimum graduation credit requirements in core subject areas (English, math, science and social sciences) and teacher salaries, both holistically and in …

Career Development And Employability Skill Integration Within The Equine Science And Management Undergraduate Program At The University Of Kentucky, Savannah Faye Robin

Theses and Dissertations--Educational Leadership Studies

College graduates need to be equipped with career and employability skills that are necessary to be meaningfully and gainfully employed in their future. These skills consist of self-management, career-management, essential-employability, and discipline-specific skills. More responsibility is being placed on college and university academic programs to ensure that students are graduating with these skills when they enter the workforce. However, students are growing less likely to participate in out of course career development activities and resources. Integrating these skills into program curriculum can be an effective way to ensure that all students have the ability to develop these necessary skills.

While …

Role Strain And Role Satisfaction Of Academic Program Coordinators At A Southern Research-Intensive Doctoral Granting University, Samer R. Jan

Theses and Dissertations--Educational Policy Studies and Evaluation

The research dissertation examined, through a self-administered survey, the activities academic program coordinators perform at a US research-intensive university and whether they suffered from role strain during the 2020-2021 academic year. The dissertation also explored what academic program coordinators found satisfying in performing their roles. The research contributes to the field of higher education by providing a better understanding of the roles and activities academic program coordinators perform, along with factors that cause role strain among program coordinators. A total of 47 program coordinators responded to the survey representing a 20% response rate.

The research found that program coordinators perform …

An Investigation Into The Relationship Between Teacher Grit, Technology Self-Efficacy, And Technology Integration, Joshua Jeremiah Shiloh Marsh

Theses and Dissertations--Education Sciences

The use of educational technology applications has grown tremendously in the last decade. Instructors are now equipped with hardware and software applications previously unavailable, such as mobile and interactive technologies. These tools can have tremendous impact on students’ learning and teacher practices. Teachers can improve their assessment capabilities through technology integration, provide better learning opportunities for students with learning disabilities, and promote deeper learning practices. Due to these benefits, budgets at the federal, state, and local levels of the United States now have specific allocations regarding technology-related purchases. Nevertheless, barriers remain regarding the effective integration of technologies in public schools. …

Comparing Self-Delivered To Instructor-Delivered Reinforcement During Vocational Instruction For Students With Intellectual Disability Using Video Activity Schedules, Katherine Elise White

Theses and Dissertations--Early Childhood, Special Education, and Counselor Education

In this study, an adapted alternating treatments design was used to compare the effectiveness of teaching vocational task when using self-delivered reinforcement versus instructor-delivered reinforcement while using video prompting. Participants consisted of four high school students who had been diagnosed with intellectual disabilities. Results indicated that instructor delivered reinforcement was slightly more effective at teaching a vocational task for 2 of the 4 participants. The results of the other 2 participants indicated that both forms of reinforcement delivery were similarly effective.

Undoing The Traditions Of Grading And Reporting, Thomas R. Guskey

Educational, School, and Counseling Psychology Faculty Publications

One silver lining from the ongoing public health crisis might be the lasting impact of two modified approaches to grading and reporting of the past year.

Pace Yourself: Impact Of Covid-19 On Patient-Centered Care Experience, Kristen Wilhite, Mikael D. Jones, Clark D. Kebodeaux

Pharmacy Practice and Science Faculty Publications

(1) Background: The outbreak of the novel coronavirus, COVID-19, forced colleges of pharmacy to implement new online learning methodologies to ensure that students could complete required courses. This transition was especially acute for laboratory simulation courses that require students to practice professional skills. This study aims to compare student assessment performance within a simulation-based laboratory course for students who completed the module prior to and after the online transition. (2) Methods: This study was a retrospective cohort comparison of student outcome performance with two distinct content delivery methods. Students were organized into two tracks at the beginning of the semester …

Exploring The Impact Of Pre-College Stem Exposure On First-Year Engineering Student Self-Efficacy, Micaha Dean Hughes

Theses and Dissertations--Science, Technology, Engineering, and Mathematics (STEM) Education

STEM (science, technology, engineering, and mathematics) outreach programs for high school students have become more widely available during the last decade in an effort to drive more graduates into the ever-growing STEM field in the United States. This study utilizes the Longitudinal Assessment of Engineering Self-Efficacy (LAESE) survey to investigate a possible relationship between specific pre-college STEM outreach programs and noncognitive predictors of success in first-year undergraduate engineering students. Students in their first semester of a first-year engineering program at a large, public research university completed the LAESE survey and were asked to identify their participation in one of three …

Is What Works Working? Thinking Evaluatively About The What Works Clearinghouse, Andrew A. Nelson

Theses and Dissertations--Education Sciences

Since the mid-twentieth century, the U.S. Department of Education has drafted and enacted policies to bridge the research-practice gap—that is, the gap between “what works” according to educational research and what is actually practiced by teachers and their administrators (e.g., Dirkx, 2006; Joyce & Cartwright, 2019; Tseng, 2012). One of the latest manifestations of this “what works” political legacy is the What Works Clearinghouse (WWC), which took shape as part of the Institute of Education Sciences (IES) in 2002. The WWC’s mission is to be a “central and trusted source of scientific evidence for what works in education” (WWC, 2020d, …