Educational Assessment, Evaluation, and Research Commons^™

Limitations Of Using Large Language Models For Automated Essay Scoring, Thomas A. Fink

Theses

Background: Automated essay scoring (AES) is a challenging deep learning problem. The two most widely used methods for predicting essay quality scores, supervised learning-based and LLM-based, have their own limitations. Although supervised learning-based methods are more accurate, they only predict a score and do not offer descriptive feedback to students. On the other hand, LLM-based methods can offer rubric-guided feedback but are known to be less accurate.

Methods: This work focuses on improving the accuracy of state-of-the-art LLM-based AES methods. We began by thoroughly investigating why these methods were performing poorly for certain datasets and certain examples. This led us …

Examination Of The Criterion-Related Validity Of The Scales For Assessing Emotional Disturbance – 3rd Edition, Corey Pierce, Jacqueline Huscroft-D'Angelo, Jessica Wery, Brad Uhing, Jodie Martin, Kara Loftin

Journal of Educational Research and Innovation

In this study, we sought to determine if the Scales for Assessing Emotional Disturbance – Third Edition (SAED-3; Epstein et al., 2020) demonstrated adequate validity using criterion validation techniques. The associations between the SAED-3 subscales and composite scores were examined in relation to similar scores from the Behavior Assessment System for Children – Second Edition (BASC-2; Reynolds & Kamphaus, 2004) and the Strengths and Difficulties Questionnaire (SDQ; Goodman, R., 1997). SAED-3 ratings were conducted with either the BASC-2 or SDQ with 121 students ages 6 to 17 years. The overall criterion prediction evidence of …

Qualitative Tools To Enhance Data Analysis From Diverse Sources Of International Education Policy In Order To Create A More Perfect Evaluation Toolset, Robert J. Schulenburg

Capstone Collection

Abstract

This project proposes a new protocol for implementing meta-synthesis as a research tool. Meta-synthesis allows researchers to combine data sets from multiple sources that appear unconnected; it allows data sets to be extracted from multiple domains and knowledge banks that present as being unable to be combined or cross-transferable. This project presents a protocol designed to guide meta-synthesis which was used to extract critical relevant data from the Sustainable Development Goals (SDGs) related to inclusive education and inclusive work and rehabilitation policies and practices. The project shows with deliverables how, with this data in hand, newly aggregated extracted data …

Let Us Break Bread Together: A Historical Analysis Of African American Chefs’ Influence On American Gastronomy, Danielle S. King

Capstone Collection

This Capstone project examines the historical impact of African American chefs on American gastronomy, highlighting their substantial contributions to the nation's culinary heritage from 1619 to 2024. By employing Jessica B. Harris's book High on the Hog as a foundation, the research identifies a critical gap in existing study abroad programs: the Black diaspora within the United States. The proposed four-week educational program aims to immerse participants in African American culinary heritage across four culturally rich cities: Charleston, South Carolina; New Orleans, Louisiana; Chicago, Illinois, and Oakland, California. With a community cultural wealth framework, the program integrates historical research, hands-on …

Navigating Uncertainty In Doctoral Education, Megan K. Mitchell, Audra Skukauskaitė, Lakelyn E. Taylor, Kathryn Babb, Nicole Narkiewicz, Joseph E. Lloyd

The Qualitative Report

The shift to more virtual learning during and since the COVID-19 pandemic has disrupted in-person doctoral education by limiting opportunities for face-to-face academic socialization, networking, and collaborative learning, prompting questions about how graduate students could sustain their scholarly development through online spaces. This study examines how graduate students who joined the Qualitative Research Club, a virtual interdisciplinary student organization, co-constructed a shared academic culture, or languaculture, through their interactions. Using an Interactional Ethnographic approach, we analyzed video and audio recordings, meeting artifacts, and discourse from club meetings to trace members' participation, identity development, co-construction of knowledge, and development of …

Increasing Understanding With Multiple Means Of Representation: Interactive Workshop For Teaching Staff, Cathy Stephens

Staff Scholarship - Australia & Dubai

Universal Design for Learning (UDL) is an educational framework aimed at optimising knowledge acquisition for all types of learners, recognising the uniqueness of each.
Within this framework, Multiple Means of Representation (MMR)—providing resources in varieties of forms—maximises access to content processing.
This forum presentation gave an overview of Universal Design for Learning concepts to lecturing staff at SAE University College, exploring some strategies used to diversify content and engage learning. It also used Padlet to engage attendees in a set of active learning exercises to examine manageable ways of incorporating multiple resources.

This National Forum workshop included collaborative, eLearning, active …

Artificial Intelligence And Transformative Curricula Practices In Africa For Educational Development, Ibiwumi Abiodun Alade, P Mthetwa

International Journal of Emerging and Disruptive Innovation in Education : VISIONARIUM

Industrial revolutions in the world have orchestrated numerous innovations, and the impact on educational landscape in different countries do not leave the society to remain static. However, with the emergence of Artificial Intelligence (AI) in the fourth era of industrial revolution, the structural shift and the effect it might significantly cause in transformative educational and curricula practices deserve attention. This paper therefore, explored “artificial intelligence and transformative curricula practices in Africa for educational development” In the paper, a review of the transformative era of the Four Industrial Revolutions (4IR) within which artificial intelligence emerges has been explored, Also, an …

Conversations In Isolation: A Mixed-Methods, Multi-Perspective Evaluation Of Extreme Electronic Service Learning During Covid-19, Lauren M. Dinour, Jennifer Y. Daclan

Journal of Community Engagement and Higher Education

Due to physical proximity constraints imposed by COVID-19, extreme electronic service learning (e-service-learning)—instruction and service offered completely online—allowed for the safe continuation of service-learning experiences. Using a combination of surveys and an autoethnographic approach, this study describes findings of an intergenerational extreme e-service-learning project from four perspectives: students, assisted living facility residents, community partner, and course instructor. Positive impacts were noted on personal, social, and learning contexts among each of the four perspectives.

A New Approach To Self-Assessment Of Community Engagement: A Case Study Of Technological University Dublin, Emma O'Brien, Martina Brophy, Thomas M. Cooney

Journal of Community Engagement and Higher Education

Building upon existing international tools, the TEFCE (Towards a European Framework for Community Engagement of Higher Education) Toolbox for community engagement represents an innovative, robust, and holistic framework to support universities in reflecting upon their community engagement. Through a case study approach, this paper highlights the piloting of the TEFCE Toolbox at Technological University Dublin (TU Dublin), Ireland. Through an inclusive and participatory methodology, the process facilitated the recognition of community engagement achievements and the identification of potential areas for improvement.

Establishing An Evaluative Mechanism For Cura Personalis, Jarrod Stadnyk

Jesuit Higher Education: A Journal

Cura personalis is an important characteristic of Jesuit education. Peter Hans Kolvenbach, S.J. helped to focus attention on the importance of cura personalis in Jesuit education. However, rather than simply being individual care for the whole person, cura personalis is an illusive concept that is easier to recite than apply in practice. This article proposes an evaluative mechanism that can be used to gauge a person’s or organization’s depth in providing cura personalis for higher education students. The Higher Education Cura Personalis Reflection Chart within this article is a new mechanism that can help guide the process of determining what …

Fy 2023-2024 Annual Report (Lwlc), Lwlc Faculty And Staff

LWLC Publications

No abstract provided.

Evidence Base And Development Of The Wellbeing Program Guide, Katherine L. Dix, Ee Liz Puah, Tamara Van Der Zant, Syeda Kashfee Ahmed, Shani Sniedze

Wellbeing

This report outlines the background and initial development stage of The Wellbeing Program Guide (wellbeingguide.acer.org), a comprehensive national resource aimed at supporting schools in selecting and implementing wellbeing programs that meet diverse contexts and student needs. Wellbeing is considered in its broadest sense, to include evidence-based mental health, alcohol and other drug, and suicide prevention programs, along with social and emotional learning. Developed collaboratively by the Australian Council for Educational Research (ACER) and Department of Health Tasmania, with advice from education and industry stakeholders, the Guide prioritised the needs of school communities in Tasmania, but also includes programs serving other …

"Inspiring Empathy": A Descriptive Review Of A Musical Composition, Garo Saraydarian

Educational Studies Dissertations

This study explored the experiences of college professors using an assessment approach, descriptive review, to assess undergraduate music compositions. College professors, and educators in general, often debate best practices around assessing student work. A literature review showed a debate amongst music educators over approaches to assessing student compositions, with rubrics being especially controversial. Descriptive review uses collective discussion and a series of non-judgmental exercises that aim for participants to become more intimate with a student and their work. As an assessment technique that had been used to look at student artwork, albeit at the elementary grade level, descriptive review has …

Evolution Of The Usu Extension Farm And Ranch Succession Program, Jacob A. Hadfield, Joshua Dallin, Ryan Larsen, Matthew Garcia

Outcomes and Impact Quarterly

The Utah State University (USU) Extension Farm and Ranch Succession Program has been operating for five years, providing critical support to agricultural producers across Utah. Although its impacts were previously documented (Dallin et al., 2021), the program has since been revised and better adapted to meet the needs of producers. This publication highlights USU Extension’s efforts to address the growing need for farm and ranch succession training and examines the program’s ongoing impacts on agricultural communities throughout the state.

The Climate Adaptation Intern Program: Training A Next-Generation Workforce While Bolstering Utah’S Capacity To Respond To Climate Change, Scott Hotaling, Kendall Becker

Outcomes and Impact Quarterly

Launched in Spring 2024, the Climate Adaptation Intern Program is a semester-long, cohort-style training program that pays undergraduates to work with experts and program leadership on developing fact sheets and other resources tailored to climate adaptation needs in Utah.

Belonging In Math, Katie Bainbridge, H. Colleen Sinclair, Andrew Burns

Reports

The presentation describes the steps taken to develop a "belonging in math" measure that can be used with elementary-aged school children. Questions examined include: (1) What is belonging? (2) What is belonging in math? (3) Are there additional considerations for diverse students? (4) What are the dimensions of belonging? and (5) What are some sample belonging items?

Exploring Teachers' Perceptions And Experiences In Adopting And Implementing High- Quality Instructional Materials At A Public Charter School In A Mid-Atlantic State, Marietta Maloy Williams

Graduate Student Dissertations, Theses, Capstones, and Portfolios

This action research dissertation examines the perceptions and experiences of public charter school teachers in a Mid-Atlantic state regarding the adoption and implementation of high-quality instructional materials (HQIM). Using a mixed-methods approach, the study investigated the impact of HQIM on teaching practices and student outcomes. A survey, followed by focus groups, gathered both quantitative and qualitative data from a targeted population of K–8 teachers. Data analysis included thematic coding of focus group discussions. Findings reveal significant challenges in HQIM implementation, highlighting the need for enhanced professional development and ongoing instructional support. The study concludes with implications for practice and recommendations …

Navigating Learning Gaps: Exploring Elementary Teachers’ Perspectives On Literacy-Based Strategies For Ells In Group Instruction, Ashley N. Banegas

Doctor of Education (Ed.D)

Teachers are to implement frequent pedagogical self-reflection to effectively differentiate instruction in whole-class and small-group settings for English language learners (ELLs). This qualitative, exploratory case study explored the perspectives of general elementary education teachers toward implementing varied instructional strategies used to target the literacy learning gap in ELLs. This current study investigated 12 kindergarten through fifth-grade teachers from a public charter school in southern Florida who participated in an interview or completed a journal entry. Information retrieved from scholarly literature and data were analyzed to answer this study’s three research questions. Findings determined the importance of teachers’ learning about their …

An Agricultural Needs Assessment In The Tri-County Area Of Utah, Cheyenne Reid, Lendel K. Narine, Brooke Holgate

Outcomes and Impact Quarterly

An agricultural needs assessment was conducted in the Tri-County area, i.e., Uintah, Daggett, and Duchesne counties, to inform extension programming. Results showed that profitability, business development, and land use were high-priority needs. Gaps in producers’ access to information were found, highlighting the need for targeted education programs to address agricultural challenges.

The Untapped Power Of Citizen Science: A Survey Of Utah Residents, Wesley Crump, Elizabeth Cohen, Shital Poudyal

Outcomes and Impact Quarterly

Citizen Science is an approach to involve the public in the scientific process to address locally relevant needs. A survey was conducted to explore Utahns’ relationship with citizen science. Results show a gap between respondents’ past participation and willingness to participate in citizen science, indicating an opportunity for USU Extension to strengthen its relationship with the public through citizen science.

Teaching Learning Newsletter Vol 5 Issue 1, Sastra Deemed To Be University

TLC Newsletter

No abstract provided.

Evaluating The Predictive Value Of Basc-3 Anxiety Scale And Working Memory Index Scores For Clinically Significant Test Anxiety: A Comprehensive Analysis Of Non-Classifiable Cases, Des'tiana S. Williams

Electronic Theses and Dissertations

This dissertation will explore how test anxiety (TA) affects working memory (WM) and whether certain factors can help predict higher levels of test anxiety. Anxiety has been defined as a common experience that can show up in various forms—like general worry, social fears, panic attacks, and specific phobias (APA, 2022). It was described as often triggering our natural “fight-or-flight” response, which can impact our ability to focus, think clearly, and remember things. For students, test anxiety has been especially challenging, affecting how they perform in high-stakes academic settings. Using data from a university based psychological assessment center, this study will …

Towards Accountability: A Primer On The Space Debris Problem And An Overview Of The Legal Issues Surrounding It, William Schonberg

Civil, Architectural and Environmental Engineering Faculty Research & Creative Works

Since 1957, the near-Earth population of trackable space objects has grown in number to over 36,000. Of these 36,000+ trackable objects now in low Earth orbit, just a few thousand are working spacecraft. The rest are Earth-orbiting objects which are no longer operational and are considered to be space junk. Because this junk can no longer receive maneuvering commands from its Earth-based owners, the survivability of other spacecraft traveling through or operating in Earth orbit can be jeopardized by the impacts of any number of pieces of this space junk, whose origins can usually be traced back to defunct satellites. …

Enhancing Educational Outcomes For English Learners: The Role Of Culturally Responsive Teaching And Professional Development In Texas, Elisa Garcia, Kristi L. Santi

Journal of English Learner Education

The increasing population of English learners (ELs) in Texas public schools requires a focused approach to their education. This article explores the challenges faced by ELs, including the risk of misplacement in educational programs and the need for culturally and linguistically responsive teaching practices. Drawing on recent research, the article highlights the importance of professional development (PD) tailored to the unique needs of ELs. Effective PD programs should prioritize cultural and linguistic awareness, collaborative learning opportunities, and strategies for differentiated instruction. The article also discusses the significance of engaging parents and the community in supporting ELs. By implementing these recommendations, …

Everything Is Bigger In Texas: English Learners And The Difficulty Within Assessment, Identification, And Program Development, Gloria A. Avila, Kristi L. Santi

Journal of English Learner Education

This report explores the critical components of effective program placement for English learners, emphasizing the need for accurate assessment and identification processes to ensure appropriate placement. It highlights the rapid growth of the EL population, particularly in the state of Texas, and underscores the importance of developing tailored educational programs that meet the unique needs of English learners. By examining current practices and identifying key areas for improvement, this paper aims to enhance the academic and linguistic outcomes for English learners, ultimately fostering an inclusive and supportive educational environment.

Why The “Whole Language Approach” To Reading Fails Ell Students & Instructional Strategies To Employ Instead, Caitlyn E. Petrus

Journal of English Learner Education

The research presented in this article demonstrates a qualitative view of the science of reading for phonemic awareness instruction, phonics instruction, vocabulary development, and reading comprehension strategies for English Language Learners (ELLs) in PK-12 schools. The research builds upon esteemed writers Dr. Sharon Vaughn and Dr. Sylvia Linan-Thompson’s work in their book Research-Based Methods of Reading Instruction for Learners, Grades K-4 and other reading experts such as Dr. Wiley Blevins and Dr. Lisa Moats to apply to the growing population for ELLs in PK-12 schools and how to shift from a “whole language classroom” to a classroom built on the …

The Use Of Phonics With Adult Ells At The Beginner Level, Jose V. Torres

Journal of English Learner Education

This study examines the work of Jose V. Torres, author of the Level 1 beginner adult ELL textbook I Want To Learn English, to evaluate the effectiveness of phonics instruction in literacy development for adult English Language Learners (ELLs). Drawing on data from two primary sources—(1) textbook-based assessments focused on students’ ability to distinguish between long and short vowels, and (2) CASAS outcome scores collected under controlled conditions at Baltimore Community College—this research addresses a significant gap in the literature on adult ELL literacy acquisition. Specifically, the study investigates the impact of explicitly integrating phonics instruction into a contextualized curriculum …

Towards A More Effective Individual Learning Account Program In Indonesia: Policy Insights From South Korea’S Tomorrow Learning Card, Reza Fajarsyah, Raphel Okfernando, M. Alfie Ridwan, Maeshita Fariska Ayuningtyas

Jurnal Vokasi Indonesia

Indonesia has implemented the Individual Learning Account system under the name Kartu Prakerja, or Pre-Employment Card, to support workforce development. This study examines its effectiveness by comparing it with South Korea’s Naeil Baeum Card (Tomorrow Learning Card). Despite its role in improving skills, Indonesia’s Individual Learning Account system faces challenges in funding sustainability, eligibility control, participant engagement, and training relevance. Additionally, financial support is provided at a fixed amount, regardless of training type. Using comparative policy and fishbone analysis, this study identifies key differences and best practices from South Korea. The findings highlight that South Korea’s co-funding model, employer …

Russian Verbs Of Motion In L2 Acquisition: Testing The Efficacy Of A Usage-Based Approach, Tobias-Alexander Herrmann

Russian Language Journal

Unprefixed Russian motion verbs present unique challenges for second language (L2) learners as they encode distinct types of motion events. Traditional, directionality-based methods of teaching these verbs have faced criticism, prompting the adoption of alternative, cognitively informed approaches. This article builds on usage-based methodologies, such as those proposed by Nesset and Janda (2022), to investigate whether teaching unprefixed verbs of motion (UVoMs) as constructions can enhance L2 instruction. Using the frequent UVoM pair idti and xodit′ , both meaning ‘walk’, as a case study, I pursue two main goals. First, drawing on corpus data, I examine the construction UVoM + …

Aligning Models Of Competency-Based Education With Skills For The Future, Teresa M. Ober, Lei Liu, David Nitkin, Ou Lydia Liu

Chinese/English Journal of Educational Measurement and Evaluation | 教育测量与评估双语期刊

Essential competencies such as collaboration and critical thinking, integral to long-term success, have been notably absent from traditional curricula, as the current instructional materials largely focus on disciplinary domains. Previous research, as highlighted by Liu, Kell, Ling et al. (2023), established a compelling link between these competencies and success across educational, occupational, civic, and health and wellbeing outcomes. The Skills for the Future, a joint initiative by ETS and Carnegie Foundation, presents a comprehensive approach to transform skills identification, demonstration, and verification. The initiative aims to create an ecosystem for skills consisting of a skills framework, assessment, extraction, and insight, …