Educational Assessment, Evaluation, and Research Commons^™

Proof Of Concept: Leveraging Large Language Models For Qualitative Analysis Of Participant Feedback, Andrew B. Crocker, Marcelo Schmidt, Juan Diego Tejeda, Howard Rodriguez-Mori

Journal of Extension

This proof-of-concept study explores the innovative application of Large Language Models (LLMs) for qualitative analysis of feedback from an Extension program, addressing the challenge of efficiently analyzing qualitative data. The study juxtaposes traditional human-led qualitative analysis with Artificial Intelligence (AI)-driven techniques, revealing the complementary strengths of human insights and AI efficiency. It underscores the potential of LLMs to enhance qualitative analysis while recognizing the need for human oversight to ensure depth and context accuracy. This research contributes to the fields of program evaluation and data analysis, offering a new paradigm for integrating advanced AI tools in qualitative research.

Workshop Wine Wednesdays: Change In Behavior Around Local Wines, Aude Watrelot, Sarah Francis, Sarah Al-Mazroa Smith

Journal of Extension

Wine workshops hosted at local wineries positively impacted the behavior of Iowa wine consumers. Many participants (75%) reported visiting an Iowa winery following the workshop. The percentage of participants consuming Iowa wines increased by 20.6% (p

Youth Public Speaking Self-Efficacy Survey Tool, Roshan Nayak, Steven M. Worker, Nicole Marshall-Wheeler

Journal of Extension

Extension has a role to play in workforce development by preparing the next generation to be successful communicators. While Extension 4-H youth programs have contributed to public speaking development since their inception, there has not been a validated instrument to assess young people’s public speaking self-efficacy. We utilized an iterative-scale development process to develop, test, and refine such a tool. We hope youth development professionals will consider using the Youth Public Speaking Self-Efficacy (PSSE) survey tool to measure and report youth’s PSSE skills, along with the sources influencing these skills, to help with program planning, implementation, and professional development activities.

Internet Addiction And Academic Fatigue, Engagement, And Performance Among Undergraduates In Southwestern Universities In Nigeria, Mensah Prince Osiesi, Olutoyin Olufunke Fajobi, Sikeade Mercy Adegboyega, Abiodun Adesope Fadiya, Olaitan Titilayo Akinola, Folasade Oluyemisi Olayinka, Muyiwa Sunday Ajimuse, Sunday Nnamdi Okocha, Tolulope Oluwatoyin Olayiwola-Adedoja, Oladipo Adeyeye Olubodun Phd, Atinuke Titilope Babalola, Valentina Grion

Higher Learning Research Communications

Objective: Internet addiction among university undergraduates may deeply impact their academic lives. This study investigated the association between academic fatigue, academic engagement, academic performance, gender, and internet addiction and how the variables jointly and relatively predict internet addiction among undergraduates in southwestern universities in Nigeria.

Method: The correlational research type of the non-experimental research design was adopted for this study. A multi-stage sampling procedure was used to select 1,157 students. The main variables were the Internet Addiction Test (r = 0.82) and the Academic Fatigue Questionnaire (r = 0.85; both adapted), as well as the Academic Engagement Scale …

Generative Ai For Synthetic Data Creation: Building Mastery-Focused Educational Datasets, Tapiwa Amion Chinodakufa, Khandaker Mamun Ahmed

SDSU Data Science Symposium

Synthetic data is artificially generated data that mimics the statistical properties of real world data without exposing sensitive information. It is used in analysis, research, and deployments. Educational technology (EdTech) is an area where synthetic data can solve the problems of data scarcity, privacy concerns, regulatory compliance, bias reduction, data quality, data integrity, and cost efficiency. Our research aims to generate synthetic educational dataset by leveraging generative AI techniques such as Autoencoder, variational autoencoder and Copula-GAN. Our experimental results shows the significant progress in generating educational dataset and represents the data distribution of synthetic and real data.

Contingent Enrollment Initiative At A Highly Selective Public University, Sean Dufault

Journal of Academic Underperformance

When first-year students arrive on campus, they may face many challenges, such as insufficient academic preparation, conflicting obligations, social adjustment, and traversing the college landscape. This challenge is magnified for academically at-risk students leading higher education institutions to invest funds and retention efforts into student success programs. The current study was designed to investigate conditionally admitted at-risk first-year students at a highly selective public university (HSPU) concerning their perceptions of participating in the Jump Start program regarding their adjustment to the university, the program's requirements, and their sense of belonging to the university. The current study employed an inductive analysis …

Research Conference 2025 - Transforming Learning Systems: Conference Abstract Book, Australian Council For Educational Research

2021-2030 ACER Research Conferences

Research Conference 2025 – Transforming learning systems, brings together international experts discussing evidence-based, high-quality approaches to transform educational practice. The focus of this year’s conference is on enabling holistic growth for all students by building sustainable capacity within education systems. The conference centres on fostering a shared language, understanding and commitment to continuous improvement in practice. Research Conference 2025 provides an opportunity to discuss practical frameworks for the complex nature of system and school improvement and building an equitable, resilient and inclusive learning system.

Use Of Simulation In Dietetic Internships: A Qualitative Exploration Of Program Director Attitudes And Motivations, Theresa B. O'Reilly, Amy J. Nwora, Megan Whelan, Mary-Pat Maciolek

Journal of Dietetic Education

The use of simulation holds promise for the future of dietetics education for its use as an alternative for up to 30% of the 1,000 supervised practice hours for the training of Registered Dietitian Nutritionists. Even more so, simulation is a teaching tool. When done well, simulation can challenge students in a safe environment where they can receive constructive feedback from expert guides and meaningfully reflect on their learning. The purpose of this study was to interview dietetic internship (DI) program directors on their attitudes and motivations on the use of simulation in their programs, targeting the heavily program-congested states …

The Greenhouse Effect: Using Student-Generated Agricultural Data To Warm Up Students For Data-Based Decision Making, Shonda Kuiper, Abhishek Chakraborty, Tyler George, Lisa W. Kay, Lawrence M. Lesser, Ginger Rowell, Dennis Pearl, Scott Crawford, Anna Olsen

EKU Faculty and Staff Scholarship

This article describes a collaborative project across multiple institutions to compare two versions of a regression activity: (a) a website version that collected student data through an online simulation with corresponding apps for visualization and (b) a traditional paper handout version that had a static dataset. This allowed us to evaluate the differences in student learning and attitudes when students have a stronger connection with the data through an interactive—and hopefully fun—data-collection process than they do when they are given a worksheet with a pre-collected dataset. While student and instructor responses indicated they liked the activity in both formats, the …

National Assessment Program Civics And Citizenship 2024. Public Report, Tim Friedman, Kate O'Malley, Wolfram Schulz, Judy Nixon, Dominique Hall, Frances Eveleigh

Civics and Citizenship Assessment

This report presents the findings of the 7th National Assessment Program – Civics and Citizenship, which was administered in May 2024, and which tests students’ civics and citizenship knowledge and understanding in a sample of Australian schools. The 2024 report reveals that Year 6 and 10 students have a high degree of trust in civic institutions, but considerably less trust in the media or social media. In addition, students have also expressed concerns about a range of issues affecting Australia, including pollution, climate change, crime, and racism. It also shows that, at the national level, results have fallen since the …

Evolving Design Critique For A Changing Industry, Sejin Hwang, Amy Ai, Leo Bang, Yuchen Chen, Kathryn Chen, Tiffany Chang, Stephen Jiang, Chloe Jo, Eileen Lee, Nicole Lu, Ray Liu, Kevin Wei, Elaine Zhang, Ellano Zhou

Student Work

This class serves as an exploratory practice in response to existing studies evaluating critique environments in the design industry. Previous studies identified three key limitations in the critique process: (1) a lack of experience for novice designers working with non-designers when reviewing their work, (2) an ineffective final review structure, and (3) a need for a more constructive and engaging feedback-sharing environment.1 To address these challenges, the course was designed to “critique” critique activities and develop strategic solutions to transform the way design work is reviewed, analyzed, and critiqued. The course was part of a pilot fall semester program called …

Secondary Stage Students' Perceptions Regarding (Myschool) Learning Management System's Role In Learning English, Huda Ali Sallam

Journal of Human Services: Training, Research, and Practice

This study set out to explore students' opinions and self-efficacy level in relation to the use of MySchool learning management system. The study has added focus on its usefulness in facilitating English language learning. The study sample involved 420 female secondary stage students within one educational district in Riyadh. The study instrument is a 24-item closed-ended questionnaire which was distributed using a Microsoft Forms link. The items of the questionnaire were divided into four main elements: perceived practicality, perceived usefulness, general satisfaction, and sense of self-efficacy. The obtained data was subsequently analyzed using Statistical Packages for Social Sciences (SPSS) 24 …

The Economic Imperatives Of Ensuring Accessible University Education In Nigeria, Adekunle Alexander Balogun

Bullion

This study exmined economic imperatives of enhancing access to university education for Nigerians. It submits that university education is the most important factor for technological and innovative breakthroughs, as it equips graduates with receptive minds for complex reasoning that innovation requires. The study employed critical discourse analysis, focusing on the association between the attainment of a university education and the higher incomes of its recipients. The study recommends that the government increase funding for university education to complement private-sector initiatives. It also recommends that university authorities in Nigeria should tap into the vast endowment and embrace grant opportunities for funding.

Toward A Critical Pedagogy Of Assessment Integrating Social Studies Education: The Proposed Socio-Critical Assessment Integration Framework (Scaif), Caroline O. Alordiah, Regina Arisi

Journal of Research Initiatives

The Socio-Critical Assessment Integration Framework (SCAIF) and its consequences for social studies education are examined in this study. The results emphasize the advantages of developing inclusive learning environments, encouraging comprehensive evaluation procedures, encouraging applicability to the real world, and giving students agency by allowing participation in the evaluation process actively. Teachers can promote social justice, question prevailing narratives, and foster critical discourse in the classroom using a critical pedagogy lens. Student learning outcomes are improved by authentic assessment activities that reflect real-world problems and demand that students use their knowledge in meaningful ways. Inclusive learning settings respect student diversity, encourage …

An Investigation Of Ninth Grade Students’ Perceptions Of Equitable Grading Practices, Sarah Ruth Morris, Sarah C. Mckenzie, Jonathan Wai, Robert Anthony Maranto

Journal of Research Initiatives

This study investigated students’ perceptions of grading equity practices, focusing on preferences and the influence of student characteristics on support for grading equity. Through surveys and open-ended questions involving 256 ninth-grade students, the research highlighted a general preference for fair and supportive grading systems that consider individual circumstances and offer retake opportunities. Multivariate OLS regression analysis reveals that while Pre-AP students show resistance, students experiencing high-stress levels favor equitable grading practices. The findings suggest the need for educational reforms prioritizing transparency and fairness, including student voices in policy development, and emphasizing grading systems that support holistic student development.

Expanding The Healthy Project: Assessing Health Literacy And Health Information Navigational Skills In High School Youth, Isaac Perez, John Ronnau, Eron Manusov

MEDI 9331 Scholarly Activities Clinical Years

Addressing the gaps identified by previous health literacy assessments in the Rio Grande Valley, this study evaluated an educational intervention aiming to enhance eHealth literacy among adolescents at a Title I high school by incorporating peer-to-peer education that targeted state educational standards for high school health science courses. Results demonstrated a statistically significant improvement in students' mean eHEALS scores by 6.56%, notably in using online health resources effectively. However, students' abilities to critically evaluate health information showed less improvement. Although statistically significant, the practical relevance of these improvements on students' real-world health-seeking behaviors requires further investigation, pointing to the necessity …

Chatgpt Didn’T Write This: Evaluating The Impact Of Llms With A Case Study In Grading Cuny Language Immersion Program Student Essays, Benjamin Inbar

Dissertations, Theses, and Capstone Projects

This study evaluates the capabilities and limitations of large language models (LLMs), specifically OpenAI’s ChatGPT-4o, in grading essays from students in the City University of New York’s Language Immersion Program. The program serves English language learners with diverse linguistic and demographic backgrounds, offering intensive language instruction to prepare students for academic success in college. Using a dataset of 30 pre- and post-program essays scored by program instructors and ChatGPT-4o under three paradigms, this research explores the alignment between human and AI-generated scores across five rubric-based competency areas. Findings reveal that ChatGPT-4o aligns moderately with human grading, with the strongest agreement …

Quantifying Educational Impacts Under The Nypd's Stop-And-Frisk Era, Matthew T. Stanton

Dissertations, Theses, and Capstone Projects

This project examines the educational and economic impacts of the New York Police Department's "Stop, Question, and Frisk" (SQF) program, implemented predominantly under "Operation Impact" from 2003 to 2022. Using data from the NYPD and the American Community Survey, the study explores correlations between high rates of stop-and-frisk encounters and reduced educational attainment among targeted demographics, primarily Black and Hispanic young men. It contextualizes the socioeconomic consequences of SQF policies, including their effect on lifetime earnings and community trust. The project highlights the limitations of data correlation, advocates for further longitudinal and qualitative studies, and suggests frameworks for mitigating inequities …

Navigating Climate Action: The River As Teacher For Eco-Literacy, Educational Regeneration And Creative Participation, Bibi (Silvina) Calderaro

Dissertations, Theses, and Capstone Projects

Cultures' engagements with land and its more-than-human elements reveal their attitudes toward these dimensions, spanning a spectrum from reciprocal to transactional practices. In time, these engagements form a body of knowledges that nurtures the culture and its land, or arrests their vital and diverse qualities. This research project aims to bring to light how the different ways of engaging with and relating to land are embedded in knowledge production, distribution and communication—in short, in general educational endeavors and their philosophies—to promote a better understanding of the principles of eco-literacy. Additionally, a purpose of this project is to provide insights into …

Teacher Retention And The Future Of Learning: Alarming Concerns About Preparing The Next Generation Of Citizens, Dejuanna M. Parker, Barbara D. Holmes

Journal of Research Initiatives

Research is robust in detailing the various reasons teachers leave the profession. There is scant literature on why teachers stay and what should be done to improve teacher retention. Teacher retention is a primary concern nationally and internationally. The inability to retain teachers has resulted in a teacher shortage and significantly affects student learning. Sorenson and Ladd (2020) This study yielded three emergent themes related to integral factors of teacher retention: changing attitudes on administrative support, workplace collegiality, and persistence of passion and purpose. Recommendations offered include encouraging administrators to leverage the passion and commitment of the educators to create …

A Prototype Online Platform For Exploring Post-Covid Academic Recovery In U.S. Public Schools: An Example From California Public Schools, Nicole Baz

Dissertations, Theses, and Capstone Projects

This project introduces a prototype online platform designed to analyze the effects of the COVID-19 pandemic on academic achievement in public schools and districts across the United States. Leveraging data from the Stanford Education Data Archive (SEDA) and the National Center for Education Statistics (NCES), the platform enables users to examine recovery rates as measured by standardized test scores. California serves as a proof-of-concept case study, demonstrating the platform's ability to explore recovery patterns in Reading Language Arts and Mathematics between 2019 and 2023. The platform provides interactive data visualizations and dashboards that facilitate analyses based on district size, resources, …

Near-Peer Student-Run Virtual Stem Summer Camp: Lessons Learned, Andrew Lee, Jeremy Reynolds, Margaret Reynolds, Freda Li, Lubjana Beshaj

Journal of Humanistic Mathematics

The United States Military Academy is uniquely positioned to contribute to our nation’s Science, Technology, Engineering, and Math (STEM) outreach initiatives. As one of the country’s most highly regarded undergraduate engineering schools, the institution receives a steady influx of Cadet near-peer mentors who volunteer to lead STEM modules. When the COVID-19 pandemic shut down in-person STEM workshops for the better part of 2020-2021, the Center for Leadership and Diversity in STEM adeptly transitioned to a virtual platform. Informed by research on STEM education and delivery, the Center successfully executed its signature summer camp event in a virtual format for two …

The Implementation Of Faculty Development In Providing Effective Feedback Utilizing Simulation, Mark Gustafson, Jessica Amos, Lisa Calderwood

Marshall Journal of Medicine

Background

Feedback is important in medical education, but various factors can impact its effectiveness. Feedback frameworks have been created to address these challenges, but simulation-based instruction of these frameworks has not been studied. This study aims to evaluate the effectiveness of instruction utilizing simulation to teach faculty how to deliver feedback.

Methods

Faculty development feedback workshops were held from October 2022 through May 2023 at Charleston Area Medical Center Institute for Academic Medicine. Each workshop consisted of a didactic session and 3 simulated scenarios during which the faculty learner gave feedback to a resident. Surveys were conducted before the workshop, …

The Impact Of The University Education And Knowledge Transfer Index On The Global Competitiveness Of The United Arab Emirates From The Perspective Of Quality Employees, Dr. Shawk Hasan Mahdi

Faculty Peer-Reviewed Papers | بحوث هيئة التدريس المحكمة

This study aimed to demonstrate the impact of the university education and knowledge transfer index on the global competitiveness of the United Arab Emirates from the point of view of quality employees. The study relied on the quantitative descriptive survey method. The tool was a questionnaire for data analysis. The study sample consisted of a group that was distributed to a sample of (330) of (director, executive director,

head of department/division, employee) who work in departments, departments, and offices of quality management and academic accreditation in accredited higher education institutions. The sample size was (360) individuals. The sample size was …

Students’ Geo-Capabilities Skills Improvement: The Role Of Differentiated Spatial Project Learning Development Using Qgis, Budi Handoyo, Alfyananda Kurnia Putra, Ramadhani Lausi Mkumbachi, Aulia Nindy Fadila Gastama

Jurnal Pendidikan Geografi: Kajian, Teori, dan Praktek dalam Bidang Pendidikan dan Ilmu Geografi

The research aims to develop a learning model integrating Differentiated Learning, Project-Based Learning, and Spatial-Based Learning, known as the Differentiated Spatial Project Learning (DSPL) model, using the Quantum Geographic Information System (QGIS). The study addresses two main research questions: (1) What are the steps of the DSPL model developed from the integration of differentiated instruction, project-based learning, and spatial-based learning with the utilization of QGIS? (2) Is there a difference in the average geo capability (GC) between classes taught using the DSPL model and those not taught using the DSPL model? To answer these questions, the Successive Approximation Model (SAM) …

How To Use Copilot And Chatgpt To Create Rubrics, Lina Eskew

Teaching Repository of AI-Infused Learning

This entry explores how AI can assist instructors in creating assignment rubrics that follow a transparency framework. By leveraging AI to generate clear and detailed criteria, instructors can eliminate ambiguity to ensure that students understand what is expected of them.

Ai Systems In Teaching And Learning: Principles And Practical Examples. Building A Path Forward, Jonathan Brazil, Suijing Yang, Fabienne Van Der Kleij

Digital learning research

This document provides guiding principles and practical examples for using AI in teaching and learning. Underpinned by a human-centred approach, the PATH principles serve as key guidance to ensure the ethical and effective integration of AI systems into teaching and learning. The PATH principles are: Promote teaching and learning; Advance wellbeing and social interaction; Train for AI literacy; and Harness AI ethically. Practical examples of AI applications are provided to highlight not only the diverse types of education-specific AI systems but also how generative AI can be employed to support teaching and learning. From adaptive learning platforms to AI-driven content …

Exploring School Teachers' Cyber Security Awareness, Experiences, And Practices In The Digital Age, R Ravichandran, Sonam Singh, P Sasikala

Journal of Cybersecurity Education, Research and Practice

This study investigates the awareness and practices of cyber security among school teachers, exploring their understanding of cyber threats, online behaviours, and response mechanisms to cyber incidents. A structured questionnaire was administered to gather data on demographic information, cyber security training, online practices, and experiences with cybercrime. The findings reveal varying levels of awareness among teachers, with many reporting limited knowledge of prevalent cyber threats such as phishing and identity theft. Despite the increasing reliance on digital tools for teaching, a significant number of respondents indicated a lack of formal training in cyber security. The study highlights the necessity for …

Empowering Teaching And Learning With Educational Technology: Literature Review, Suijing Yang, Daniel Taylor-Griffiths, Fabienne Van Der Kleij, Pauline Taylor-Guy, Ralph Saubern

Digital learning research

Many existing reviews of educational technologies focus on the affordances of specific types of technology rather than how different technologies can be designed and used to achieve specific teaching and learning objectives. Furthermore, there appears to be a widely held assumption that the use of educational technology will result in improved outcomes. However, there has been limited scrutiny of evidence about the extent to which use of educational technologies impacts teaching and learning outcomes in school contexts. To address these gaps in understanding of educational technologies, this literature review examines the design of technologies, exploring how they have been leveraged …

Efektivitas Full Body Stretching Dan Edukasi Manual Handling Digital Terhadap Keluhan Musculoskeletal Disorders Pekerja Yantek, Nadia Rosalinda, Anita Sulistyorini, Marji Marji, Supriyadi Supriyadi, Tika Dwi Tama

National Journal of Occupational Health and Safety

Occupational Diseases (PAK) in the form of pain complaints in the skeletal system that cause the work process are called Musculoskeletal Disorders (MSDs). PT. PLN Yantek workers are at risk of experiencing MSDs complaints because the work is done manually and the work posture is less ergonomic. This study aimed to analyze the effect of implementing full-body stretch and digital-based manual handling education and to assess its effectiveness on MSDs. The study used a quasi-experimental method with a one-group pretest-posttest design involving 26 Yantek workers. The intervention was given at 10 meetings, MSDs complaints were measured using the Nordic Body …