Educational Assessment, Evaluation, and Research Commons^™

The Human Complementary Usage Of Ai And Ml For Fair And Unbiased Educational Assessments, Hayri Eren Suna, Mahmut Özer

Chinese/English Journal of Educational Measurement and Evaluation | 教育测量与评估双语期刊

In recent years, artificial intelligence (AI) and machine learning (ML) algorithms have played an influential role in advancing educational assessment. As a means of improving equal opportunities in education, assessing students' learning deficiencies and developing personalized learning suggestions is considered an important aspect. Furthermore, big data-based algorithms play an increasing role in assessing students' cognitive and social-emotional development and conducting research on school-based monitoring. A number of these developments, on the one hand, provide valuable feedback to both students and educational authorities regarding assessment of student growth, and on the other hand, they raise the issue of bias in the …

Uncovering Dimensions Of Learning Management System Event Logs: Insights From Exploratory Factor Analysis, Psychometric Network Analysis, And Consensus Hierarchical Clustering, Surina He, Okan Bulut, Ying Cui, Tarid Wongvorachan

Chinese/English Journal of Educational Measurement and Evaluation | 教育测量与评估双语期刊

Log data extracted from learning management systems (LMS) have become an increasingly valuable resource for understanding students’ learning behaviors. These data are unobtrusive, cost-effective, and offer fine-grained insights. However, their complexity and large volume present substantial challenges in identifying interpretable underlying dimensions across different event types. To address this challenge, this study applied exploratory factor analysis, psychometric network analysis, and consensus hierarchical clustering to analyze 122,167 log entries from 25 event types collected in a large undergraduate course (n = 527 students). Four distinct clusters of student activities were identified in the event logs: submission, quiz-taking, discussion, and background-viewing. …

Toward Better Digital Tool Designs To Assist Students On Math Assessments, Hongwen Guo, Matthew S. Johnson, Luis Saldivia, Michelle Worthington

Chinese/English Journal of Educational Measurement and Evaluation | 教育测量与评估双语期刊

Digital tools in digitally based math assessments are designed to help students solve math problems more effectively. However, inappropriate use of these tools can lead to negative outcomes. Using process data from NAEP, we generalized the differential item functioning methodologies and conducted a systematic investigation of the effectiveness of tool uses. Three measures were proposed to evaluate the effectiveness of a digital tool on an item: its popularity (the proportion of students who used it), its impact on item score (the score gain/loss after matching on performance score and other tool uses), and its impact on item response time (the …

The Learning Crisis In The United States Three Years After Covid-19, Harry Anthony Patrinos, Maciej Jakubowski, Tomasz Gajderowicz

Education Reform Faculty and Graduate Students Publications

The COVID-19 pandemic caused widespread disruptions to education, with school closures affecting over one billion children. These closures, aimed at reducing virus transmission, resulted in significant learning losses, particularly in mathematics and science. Using United States data from TIMSS, this study analyzes the impact of school closure on learning outcomes. The losses amount to 0.36 SD for mathematics and 0.16 SD for science. The declines are similar across grades. The average decline in mathematics performance among U.S. students is substantially greater than the global average. n science, the decline observed among U.S. students does not significantly differ from the global …

Accessibility Disclosure Statements: Best Practices For Documenting Inclusive And Accessible Web-Based Research Design, Alicia Mason, Elizabeth Spencer, Pan Liu, Kristen Livingston, Sakshi Bhati, Angela Ashmore, Tristan A. Spencer, Lauren Shepard

Faculty Submissions

Accessibility disclosure is a core principle of the Project IRIS Best Practice Framework. This instructional resource-- Accessibility Disclosure Statements: Best Practices for Documenting Inclusive Web-Based Research Design-introduces options for reporting accessible web-based research design efforts in scholarly and professional publications. The three forms of accessibility disclosure statements are introduced: summary statements, reports of analysis--both general and specific-, and detailed process descriptions. Each of these strategies including terms, conceptual definitions, and examples are provided. The instructional resource may be of interest to undergraduate and graduate course instructors teaching research methods courses in a variety of social science disciplines.

Differences In Reading College Readiness By The Language Status Of Texas High School Students: A Statewide, Multiyear Analysis, Lidia Garza Calderón, John R. Slate, Tershundrea Branch, Clare A. Resilla

Journal of Educational Leadership in Action

The extend to which differences were present for Emergent Bilingual students and non-Emergent Bilingual students in their performance on the Texas state-mandated English I End-of-Course exam for the 2018-2019, 2020-2021, and 2021-2022 school years was examined. Specifically addressed was the extent to which Emergent Bilingual students and non-Emergent Bilingual students differed in their performance on three Grade Level performance standards: Approaches Grade Level standard, Meets Grade Level standard, and Masters Grade Level standard in the pre-pandemic year and the first and second-post-pandemic years. Non-Emergent Bilingual students had statistically significantly better English I End-of-Course exam performance than Emergent Bilingual students in …

Creating A School Climate Of Belonging And Engagement Through Visionary Leadership And Student Art, Jeff Irvine, Kyra Kristensen-Irvine

Journal of Educational Leadership in Action

This research adopted a case study methodology to investigate how student artwork helped establish a school climate that was intentionally inviting to students and the wider school community. Through the visionary leadership of two principals, a Kindergarten to Grade 5 school and a middle school enhanced their common areas through student visual arts projects. Previous research on school physical environments focused on safety, cleanliness, and noise levels. This paper addresses a research gap, identifying the role that physical environments played in promoting a positive school climate which in turn promoted students’ sense of belonging and engagement.

Algebra I End-Of-Course Exam Differences By The At-Risk Status Of Texas Hispanic Students: A Multiyear Statewide Investigation, Daniel Craft, John R. Slate

Journal of Educational Leadership in Action

Examined in this investigation was a comparison of the Algebra I End-of-Course exam performance between Hispanic students who were at-risk and Hispanic students who were not at-risk in the 2016-2017, 2017-2018, and 2018-2019 school years. Inferential statistical procedures yielded statistically significant differences in all three school years between these two groups of Hispanic students. In all three school years, higher percentages of Hispanic students who were at-risk did not meet the three grade level standards (i.e., Approaches Grade Level, Meets Grade Level, and Masters Grade Level) than Hispanic students who were not at-risk. Moreover, Hispanic students who were at-risk correctly …

Novice School Prinicpals' Culturally Responsive Leadership Practices, Maria De Lourdes Viloria, Alexandra Vasquez, Georgina Hernandez

Journal of Educational Leadership in Action

This paper investigates the perspectives of three novice school leaders who explore ways to accommodate new instructional practices at their campuses by setting high expectations and getting to know students and their educational needs. The three novice school leaders who participated in this investigation emphasized the importance of communication and relationships with all the stakeholders, including: teachers, staff, students, parents, office personnel, and community members. Additionally, all three novice school leaders expressed gaining trust and proactively providing support to teachers at all times. Moreover, they concurred that the key skills required for success as a novice school leader are: communication, …

Journal Of Educational Leadership In Action, Volume 9, Issue 3, Full Issue

Journal of Educational Leadership in Action

No abstract provided.

The History Matters: Fighting For Equal Education In Lowell, Massachusetts, 1970-1990, Robert Forrant, Chandaranee Khoeun, Chrisna Khuon

Institute for Asian American Studies Publications

This is a story about Southeast Asian and Latino parents in Lowell, MA who confronted the Lowell School Committee in the late 1970s and 1980s, demanding that schools be desegregated and that their children have equal educational opportunities. While the school struggle unfolded, acts of anti-Asian and anti-Latino violence were perpetrated in Greater Boston and around the country, including the tragic death of Vandy Phorng, a 13-year-old Cambodian American in 1987. Following his death, the federal lawsuit that Southeast Asian and Latino parents had filed moved through the federal court for two years before the city settled. However, further racism …

Evaluating Learning Outcomes In A Serious Game: A Practical And Model Checking Approach, Matthew D. Douglas

Theses and Dissertations

This research introduces a novel computational framework to evaluate and predict the educational impact of serious games during development. By using finite state machines (FSM) and model-checking techniques, this study evaluates two serious games. Traditional evaluation approaches, often reliant on resource-intensive human trials, lack scalability and fail to provide early insight into the alignment of game mechanics with learning objectives. This study addresses these challenges of traditional evaluation methods.

2024-2025 Common Data Set, Systems Intelligence And Data Analytics, University Of Nebraska At Omaha

Common Data Sets

The Common Data Set (CDS) initiative is a collaborative effort among data providers in the higher education community and publishers as represented by the College Board, Peterson’s, and U.S. News & World Report. The combined goal of this collaboration is to improve the quality and accuracy of information provided to all involved in a student’s transition into higher education, as well as to reduce the reporting burden on data providers.

Traditional Vs. Modern-Grading Practices: A Comparative Case Study, Sarah Ruth Morris, Matt Townsley

Journal of Educational Leadership in Action

This comparative case study explores the influences of traditional and modern grading systems on ninth-grade students' academic and psychological outcomes. Employing quantitative and qualitative methods, including OLS regressions and focus group discussions, our research investigates student attitudes towards grading practices at two contrasting schools—one utilizing traditional grading methods and the other a more modern, progressive, standards-based approach. Findings suggest that while modern grading practices are favored for their fairness and focus on mastery, they do not significantly enhance academic outcomes or student well-being in the short term. The study highlights the complex implications of grading reforms and underscores the need …

Assessing Teachers' Competence In Sustaining The Use Of Ict Tools For Instructional Delivery During The New Normal Era, Yinusa Faremi, Ramota Ajoke Rasaq Ms, Ramota Ajoke Rasaq, Charles Silvane, Yemisi Olubunmi Fashoto

Journal of Educational Leadership in Action

Amidst the post-covid era, blended teaching has become increasingly prevalent worldwide, necessitating educational system changes. This study aimed to assess teachers' competence in using Information and Communication Technology (ICT) tools for instructional delivery during the new normal era. The study utilized a quantitative survey design with a sample size of 150 secondary school teachers from private and public in Osun State, Nigeria. Data were collected using a structured questionnaire. The study revealed that most respondents lacked adequate competence in sustaining the use of computers to assess learners and in utilizing PowerPoint presentations during the teaching-learning process in the new normal …

Board-Head Relationships: The Role Of Relational Trust And Intentional Care In Fostering Head Of School Well-Being, Cassidy Klutts, Albert Cheng

Education Reform Faculty and Graduate Students Publications

Given the challenges of school leadership and high rates of school leader attrition, scholars have increasingly given attention to the wellbeing of heads of schools. However, that research has not considered the way governing boards potentially influence head of school wellbeing. We fill this gap by examining how relational dynamics between heads of school and governing boards are associated with head of school wellbeing. Using data from about 140 heads of schools who participated in the Society for Classical Learning’s 2023 Thriving Schools Survey, we find that (a) relational trust between heads and boards and (b) care of the head …

Widening The Net: How Cogat And Act Aspire Compare In Gifted Identification, Al Mansor Helal, Jonathan Wai, Andy Parra-Martinez, Sarah Mckenzie, Dustin Seaton

Education Reform Faculty and Graduate Students Publications

Previous research indicates that many academically accomplished students from disadvantaged backgrounds are not identified for gifted and talented (G/T) programs. This study examines a large sample of students (N = 10,508), many of whom took both the Cognitive Abilities Test (CogAT) and the ACT Aspire test. We examined test similarities and differences with an eye to widen the net for G/T identification in practice. This study demonstrates that the ACT Aspire and CogAT have a significant correlation in our sample of r = .59. However, the correlation varies across cohorts noticeably from r = .72 to r = .49. This …

Teachers’ Perspectives On Incorporating Moodle In A Ninth-Grade Reading Classroom, Cathy Miamba

Electronic Theses and Dissertations

Effective technology use in the classroom can help stimulate a variety of learning styles and engage students through fun and exciting activities that help the students comprehend the content. Although research reported the benefits of technology in the classroom, there is a lack of knowledge of the effects of technology like Moodle on students’ academic scores, engagement, motivation, and overall achievement. As such, the problem addressed in this study was the lack of effective use of technology that affects students’ motivation, engagement, and achievements in the classroom. Using Vygotsky’s constructivism theory as the theoretical foundation and qualitative case study as …

A Quantitative Study: The Association Between College Selection Methodology And Sense Of Belongingness Among First-Year, Full-Time Residential College Students, Tamara Reese

Doctoral Dissertations and Projects

College selection methodology can be traced back to the 1920s when colleges had to rely on their to travel through word of mouth, catalogs, pictures, and brochures. Although virtual tours can be dated back to 1994, COVID-19 catapulted the popularity of traveling through virtual reality. While many studies have sought to measure the impact of visitation on student enrollment propensities, none have made a noteworthy effort to account for a potential endogenous link between the student's college choice methodology and a student's sense of belongingness at their college or university campus. The purpose of this quantitative quasi-experimental study is to …

Feeding And Swallowing Disorders In Children With Autism: What Caregivers Need To Know, Olivia Hopkins

Undergraduate Publications

This study investigates the complex topic of feeding and swallowing disorders in children with autism, highlighting the significant challenges they pose, particularly related to sensory sensitivities and oral motor difficulties, and the persistent lack of understanding. We examine diagnostic criteria, risks, familial impacts, cultural nuances, and evidence-based intervention strategies, with a focus on practical guidance for caregivers, to equip them with resources for navigating these challenges (Schulz & Eden, 2016). Despite the prevalence of these disorders, many remain unaware of the struggles, underscoring the urgent need for targeted investigations and supportive measures. Specifically, this paper aims to provide Speech Language …

Sdg 4.1.1 Minimum Proficiency Levels: Definition And Blueprint For Assessment [Presentation], Colin Watson, Ursula Schwantner

Monitoring Learning

A critical element for aligning assessments to the UIS eligibility criteria is mapunderstanding minimum proficiency levels (MPLs) in reading and mathematics. The Global Education Monitoring (GEM) Centre, in collaboration with the UIS, has defined clear MPL benchmarks and developed a blueprint for assessment for the 3 educational stages under SDG 4.1.1. This presentation shared the definition and blueprint with key stakeholders at the 11th meeting of the Global Alliance to Monitor Learning in Paris from 25-26 February 2025, to ensure a shared understanding of the criteria needed for meaningful global reporting.

Building Education System Resilience In Samoa: Exploring Systems, Policies And Classroom Practices, Yung Nietschke, Anna Dabrowski, Nora Warren, Michelle Arasi-Mulitalo

International Education Research

This report presents the findings of research on education system resilience in Samoa during and immediately following the COVID-19 pandemic in order to build a deeper understanding of policies and practices in place, and what lessons can be drawn to support system-wide adjustments to mitigate future school disruptions. Data included case studies of practice, student assessment data and secondary data analysis of the 2018-2021 PILNA surveys for Samoa. The study identified the policies and practices that contribute to the Samoan education system's capacity to respond and recover from crisis and the factors that can drive system resilience, including social, human …

Youth Public Speaking Self-Efficacy Survey Tool, Roshan Nayak, Steven M. Worker, Nicole Marshall-Wheeler

Journal of Extension

Extension has a role to play in workforce development by preparing the next generation to be successful communicators. While Extension 4-H youth programs have contributed to public speaking development since their inception, there has not been a validated instrument to assess young people’s public speaking self-efficacy. We utilized an iterative-scale development process to develop, test, and refine such a tool. We hope youth development professionals will consider using the Youth Public Speaking Self-Efficacy (PSSE) survey tool to measure and report youth’s PSSE skills, along with the sources influencing these skills, to help with program planning, implementation, and professional development activities.

Proof Of Concept: Leveraging Large Language Models For Qualitative Analysis Of Participant Feedback, Andrew B. Crocker, Marcelo Schmidt, Juan Diego Tejeda, Howard Rodriguez-Mori

Journal of Extension

This proof-of-concept study explores the innovative application of Large Language Models (LLMs) for qualitative analysis of feedback from an Extension program, addressing the challenge of efficiently analyzing qualitative data. The study juxtaposes traditional human-led qualitative analysis with Artificial Intelligence (AI)-driven techniques, revealing the complementary strengths of human insights and AI efficiency. It underscores the potential of LLMs to enhance qualitative analysis while recognizing the need for human oversight to ensure depth and context accuracy. This research contributes to the fields of program evaluation and data analysis, offering a new paradigm for integrating advanced AI tools in qualitative research.

Workshop Wine Wednesdays: Change In Behavior Around Local Wines, Aude Watrelot, Sarah Francis, Sarah Al-Mazroa Smith

Journal of Extension

Wine workshops hosted at local wineries positively impacted the behavior of Iowa wine consumers. Many participants (75%) reported visiting an Iowa winery following the workshop. The percentage of participants consuming Iowa wines increased by 20.6% (p

Internet Addiction And Academic Fatigue, Engagement, And Performance Among Undergraduates In Southwestern Universities In Nigeria, Mensah Prince Osiesi, Olutoyin Olufunke Fajobi, Sikeade Mercy Adegboyega, Abiodun Adesope Fadiya, Olaitan Titilayo Akinola, Folasade Oluyemisi Olayinka, Muyiwa Sunday Ajimuse, Sunday Nnamdi Okocha, Tolulope Oluwatoyin Olayiwola-Adedoja, Oladipo Adeyeye Olubodun Phd, Atinuke Titilope Babalola, Valentina Grion

Higher Learning Research Communications

Objective: Internet addiction among university undergraduates may deeply impact their academic lives. This study investigated the association between academic fatigue, academic engagement, academic performance, gender, and internet addiction and how the variables jointly and relatively predict internet addiction among undergraduates in southwestern universities in Nigeria.

Method: The correlational research type of the non-experimental research design was adopted for this study. A multi-stage sampling procedure was used to select 1,157 students. The main variables were the Internet Addiction Test (r = 0.82) and the Academic Fatigue Questionnaire (r = 0.85; both adapted), as well as the Academic Engagement Scale …

Generative Ai For Synthetic Data Creation: Building Mastery-Focused Educational Datasets, Tapiwa Amion Chinodakufa, Khandaker Mamun Ahmed

SDSU Data Science Symposium

Synthetic data is artificially generated data that mimics the statistical properties of real world data without exposing sensitive information. It is used in analysis, research, and deployments. Educational technology (EdTech) is an area where synthetic data can solve the problems of data scarcity, privacy concerns, regulatory compliance, bias reduction, data quality, data integrity, and cost efficiency. Our research aims to generate synthetic educational dataset by leveraging generative AI techniques such as Autoencoder, variational autoencoder and Copula-GAN. Our experimental results shows the significant progress in generating educational dataset and represents the data distribution of synthetic and real data.

Contingent Enrollment Initiative At A Highly Selective Public University, Sean Dufault

Journal of Academic Underperformance

When first-year students arrive on campus, they may face many challenges, such as insufficient academic preparation, conflicting obligations, social adjustment, and traversing the college landscape. This challenge is magnified for academically at-risk students leading higher education institutions to invest funds and retention efforts into student success programs. The current study was designed to investigate conditionally admitted at-risk first-year students at a highly selective public university (HSPU) concerning their perceptions of participating in the Jump Start program regarding their adjustment to the university, the program's requirements, and their sense of belonging to the university. The current study employed an inductive analysis …

Research Conference 2025 - Transforming Learning Systems: Conference Abstract Book, Australian Council For Educational Research

2021-2030 ACER Research Conferences

Research Conference 2025 – Transforming learning systems, brings together international experts discussing evidence-based, high-quality approaches to transform educational practice. The focus of this year’s conference is on enabling holistic growth for all students by building sustainable capacity within education systems. The conference centres on fostering a shared language, understanding and commitment to continuous improvement in practice. Research Conference 2025 provides an opportunity to discuss practical frameworks for the complex nature of system and school improvement and building an equitable, resilient and inclusive learning system.

Use Of Simulation In Dietetic Internships: A Qualitative Exploration Of Program Director Attitudes And Motivations, Theresa B. O'Reilly, Amy J. Nwora, Megan Whelan, Mary-Pat Maciolek

Journal of Dietetic Education

The use of simulation holds promise for the future of dietetics education for its use as an alternative for up to 30% of the 1,000 supervised practice hours for the training of Registered Dietitian Nutritionists. Even more so, simulation is a teaching tool. When done well, simulation can challenge students in a safe environment where they can receive constructive feedback from expert guides and meaningfully reflect on their learning. The purpose of this study was to interview dietetic internship (DI) program directors on their attitudes and motivations on the use of simulation in their programs, targeting the heavily program-congested states …