Educational Assessment, Evaluation, and Research Commons^™

Meta-Theoretic Competence For Computational Agent-Based Modeling, Hillary Swanson, Uri Wilensky

Instructional Technology and Learning Sciences Faculty Publications

In the U.S. context, science standards encourage educators to engage students in modeling practices, including computational modeling. While much work has investigated the productivity of computational modeling with respect to students' development of scientific content knowledge, less work has focused on students' development of knowledge and skills for participation in computational modeling practices. A first step in understanding how these practices develop is examining students' activity in the context of computational modeling environments with attention to the productive moves they make. These moves can provide insight into the knowledge they bring to their learning, which may be foundational to the …

Design Forms And Games, Hillary Swanson, Ha Nguyen, Caroline Lavoie, Trilby Nelson

Instructional Technology and Learning Sciences Faculty Publications

This paper introduces a theoretical framework for characterizing design processes as design games. Based on Collins and Ferguson's epistemic forms and games, the framework characterizes design processes as games played by designers to fill out design forms, which are general templates underlying specific design artifacts. We introduce our framework through games involved in human-centered design, describing for each the entry conditions, target form, constraints, and moves. We demonstrate how the framework can be used to characterize design games played by experts in museum exhibit design and urban design. Through similar analyses, the framework …

The Role Of Generative Ai In Qualitative Research: Gpt-4'S Contributions To A Grounded Theory Analysis, Ravi Sinha, Idris Solola, Ha Nguyen, Hillary Swanson, Luettamae Lawrence

Instructional Technology and Learning Sciences Faculty Publications

We present reflections on our experience using a generative AI model in qualitative research, to illuminate the AI's contributions to our analytic process. Our analytic focus was a segment of classroom transcript, which captured a teacher introducing scientific theory-building practices to middle school students. We used a grounded theory approach to produce a fine-grained characterization of the teacher's talk moves during the lesson implementation. Our eventual goal is to build a more nuanced conceptualization of responsive teaching in the context of theory-building activities. We involved GPT-4 during the initial exploratory and later focused coding stages. For our analysis of GPT-4's …

What Happened To The Geometry? Examining Spatial And Mathematical Concepts In Computational Toys And Kits For Young Children, Jody Clarke-Midura, Jessica F. Shumway, Anahita Ashineh, Alireza Zandi, Whitney Clawson

Instructional Technology and Learning Sciences Faculty Publications

This paper explores the design of computational toys and kits for young children (ages 4-7) as tools for learning integrated mathematical, spatial, and computational thinking concepts. Specifically, we examine how the design features of the toys and kits represent the concepts of rotation on a point and spatial orientation of the agent. We examine toys and kits sold commercially, developed through research, and used in early childhood classrooms. Our findings indicate that the mathematical and spatial concepts are overlooked in some designs. Prior research examined toys for their affordances related to computational thinking, the present study contributes to understanding of …

Redefining Science Education Paradigm—By Course Habitable Earth, Nianzhi Jiao, Sanzhong Li, Chen Hu, Zhonghe Zhou, Degan Shu, Shucheng Xie, Jianping Huang, Fahu Chen, Lin Ding, Tong Zhu, Yigang Xu, Yonglong Lyu, Zhimin Jian, Pinxian Wang

Bulletin of Chinese Academy of Sciences (Chinese Version)

In a rapidly evolving world defined by unprecedented challenges, the integration of education, technology, and talent development has emerged as a national strategic imperative. Cultivating interdisciplinary and innovative scientific talents is not only a demand of the era but also a cornerstone for transforming “population dividends” into “talent dividends”. Against this backdrop, the rapid advancement of artificial intelligence (AI) offers both challenges and opportunities for redefining traditional education models. This study explores the pedagogical practices of the Habitable Earth general science education course, positioning it as a case study to demonstrate how science education can synergize technological empowerment with talent …

Predicting Performance On A Social Studies Assessment: A Correlational Study, David G. Hamilton

Doctoral Dissertations and Projects

This study seeks to determine the predictive relationship between the performance on a middle school Civics practice assessment and performance on the State of Florida Civics assessment. In the current public-school environment across the United States, using high-stakes assessments to measure student performance is the norm. This quantitative correlation study strives to determine if locally developed benchmark assessments accurately predict student achievement on school high-stakes social studies state-mandated assessments. Using post hoc data from 355 middle school students ranging from 12 to 14 years old from a suburban Central Florida charter school who were administered a computer-based Civics practice end-of-course …

Student Perceptions: How Stem Students Value Belonging Interventions, Elizabeth Runkel Baez

University Honors Theses

With the prioritization of increasing Science, Technology, Engineering, and Mathematics (STEM) graduates in the United States, researchers have become increasingly interested in how to promote STEM students' sense of belonging. Sense of belonging is considered a marker of persistence in STEM. Facilitated 'belonging interventions' are one way to strengthen a sense of belonging amongst STEM students. How students respond to, value, and understand these belonging interventions is relatively understudied. In this study, focus groups were conducted with STEM students from a single university to investigate how they value an in-class belonging activity. Qualitative thematic coding of focus group responses revealed …

Analyzing Common Mistakes In First-Year Medicine Faculty Students’ Paragraph Writing, Pervaiz Yaseeni, Fahim Rahimi, Fareed Sahil

Journal of Research Initiatives

Academic writing is a salient skill for medical students all over the world. Academic writing starts with composing effective paragraphs, laying the foundation for students' advancement in writing during college. This study investigated common mistakes in English paragraph writing committed by the first-year medical students of the Medicine Faculty at Kabul University of Medical Sciences. Additionally, it measured the challenges that Medicine Faculty students tackled in paragraph writing. This quantitative study was conducted through an action research design. The data were collected from 52 medical students, and their work was analyzed using the developed rubrics. The data were analyzed in …

Theory-Based Science Communication At Comic Cons, Lisa Lundgren, Emily Slater, Man Zhang, Kadie Kunz, Gabriel-Philip Santos

Instructional Technology and Learning Sciences Faculty Publications

The majority of research concerning science communication happens in spaces where people already have a vested interest in or knowledge of science, such as museums and science cafes. Thus, there is a gap in understanding what theory-based science communication looks like in non-science-centered spaces. This qualitative research study, which featured cosplaying science communicators at comic conventions, offers insights into what science communication practices occur within everyday spaces. We conducted observations of and semi-structured interviews with 15 cosplaying scientists, examining how they utilized effective framing and narrative structuring when communicating science at comic cons in cosplay. Across 700 coded utterances, cosplaying …

The Impacts Of The Higher Education Expansion On Education Quality In China : Evidence From Shenzhen University, Yousong Lun

Lingnan Theses (Doctoral degree)

Within the expansion of higher education in China, education quality should be considered into the process of reform and establishment, which is highly related to the resources of faculty and performances of management. It is crucial that universities sustainably conduct diverse academic research and meet the needs of community. Faced with the policy reform and opening up, China has more opportunities on cooperating with oversea education institutions and taking in overseas management system. This research explores the general quality assessment on higher education and the solution to balance the needs of domestic society and the growth of international education competitiveness, …

Who Gets To Decolonize: A Reflection On The Importance Of Positionality In The Decolonization Of Digital Learning Spaces And Learning Design, Rebecca Y. Bayeck, Tutaleni Iita Asino

Instructional Technology and Learning Sciences Faculty Publications

Decolonization as a term and concept gained prominence in the 20th century as new nations emerged from the cessation of the global colonial project which primarily saw European countries occupying different parts of the world. In modern times, interests in decolonization have taken different forms, particularly in learning design and digital learning spaces. In the digital space, researchers mainly suggest introducing indigenous perspectives and cultural lenses in the design of digital learning environments and the design of emerging technologies that embed indigenous perspectives (Nede, 2023). This reflexive essay discusses the importance of positionality in digital learning spaces and learning/instructional design. …

Understanding Non-Designers' Practices And Processes In A Human-Centered Design Course, Luettamae Lawrence, Saadeddine Shehab, Mike Tissenbaum

Instructional Technology and Learning Sciences Faculty Publications

Human-centered design (HCD) has been identified in the literature as a useful problem-solving approach for learners. However, learning and applying HCD poses several challenges for students who are unfamiliar with this form of learning. In this paper, we analyse how a novice design team worked on a HCD class project to examine how non-designers learn about and integrate HCD practices into their project. We introduce the HCD taxonomy to define the processes and practices that students engage in. The team’s design work is triangulated across multiple data sources and revealed three challenges for engaging non-designers in HCD: 1) the need …

Medmicromaps, A Novel Decision-Tree Guide For Infectious Diseases Differential Diagnoses, And Evaluation Of Pre- And Post-Pandemic User Engagement By Preclinical Medical Students, Jason Ceballos, Tavsimran Luthra, Lucia Garces-Torres, Valerie Lentz, Jack Nelson, Claudia Carceles-Roman, Ian Holyoak, Jane Harrington

International Journal of Emerging and Disruptive Innovation in Education : VISIONARIUM

Background: Medical education has undergone major changes in the past 5 years to adapt to the digital-centered student population with diverse learning preferences. To address the challenges of shifting delivery modalities, multimedia resources were created to provide interactive e-learning material on infectious diseases for year 2 medical students.

Methods: Medical students at a USA-accredited Caribbean medical school were provided with supplemental e-learning materials including animations, illustrations, diagnostic algorithms aka MedMicroMaps, and Case-Based tutorials. Participants were classified by semester and lecture delivery: In-person: Cohort 1 (n=526); Virtual: Cohort 2 (n=651); and Hybrid: Cohort 3 ( …

Community Engagement In Art & Design Colleges, Mah Rukh Raja

Masters Theses

This thesis titled ‘Community Engagement in Art & Design Colleges: Case Studies from Pakistan’ highlights the importance of community-engaged practices in education, specifically within the realm of art and design, and examines the effect of community engagement on student learning. It focuses on the importance of creating opportunities for reciprocal, community-engaged learning where responsibilities, roles and credit are shared equally among students, community partners, and educational institutions to develop projects that are in the best interest of communities.

This research coincides with the introduction of community and civic engagement in undergraduate education in Pakistan. Its primary objective is to serve …

Gender Disparity In Students’ Performance Of Science And Mathematics: Evidence From Nationwide Study In Pakistan, Sadia Muzaffar Bhutta, Aisha Naz Ansari, Sohail Ahmad

Institute for Educational Development, Karachi

Gender disparity in science and mathematics performance is being researched internationally. In a complex education system, like Pakistan, studying the difference in isolation may not generate holistic understanding. Therefore, this paper investigates changes in trends of performance in Science and Mathematics for both genders across public and private schools. Data were gathered from a nationwide sample of elementary grades students (n=15,391) using SATs/MATS. Independent t-test and two-way-ANOVA were employed to analyse data. Results revealed that girls outperformed boys in Science (p<0.01) while no gender differences in performance were observed in Mathematics. Gender and school systems demonstrated significant interaction effect only in mathematics, unveiling that girls perform better in public schools while this difference diminished in the private school system. The results show implications for employing gender-fair policies and practices in science and mathematics education.

An Examination Of The Relationship Between Instructional Modality And Pedagogy Licensure Exam Scores In Post-Baccalaureate Educator Preparation Programs In Indiana, Kelly Eckel

Dissertations

The efficacy of the modality of instruction employed in higher education institutions has been a point of contention among students, faculty, administration, and other stakeholders in the educational realm. Since the COVID-19 pandemic, online and hybrid instruction has become a valued and desirable option for Educator Preparation Programs (EPPs) to offer for recruitment and retention purposes. The effectiveness of varying modalities in relation to teacher licensure exams has been limited in research and can serve as a critical quantitative measure for EPPs to assess. This longitudinal study examined the relationship between instructional modality and pedagogy teacher licensure exam scores in …

A Program Evaluation Of 3de Schools By Junior Achievement, Audra Lewis-Hoesch

Dissertations

3DE Schools by Junior Achievement is a framework for instruction designed to reshape the instructional ecosystem through a high school model that uses project-based learning and case-study methodology to help students acquire core competencies. The context of this inquiry includes various states in America where 3DE Schools are implemented. I utilized a mixed methods approach to collect data and gauge the influence of the 3DE instructional framework on students' readiness for college and career pursuits. Key outcomes showed that students in 3DE cohorts scored higher in math and English than the national average, contributing to an overall higher graduation rate …

Estimating And Applying Parameters Necessary To Plan Cluster Randomized Trials (Crts) And Multisite Cluster Randomized Trials (Mscrts), Dea Mulolli

Dissertations

Cluster randomized trials (CRTs) are commonly used to study the effectiveness of educational interventions. During the design phase of a study, it is critical for researchers to ensure their studies are adequately powered to detect meaningful treatment effects, including both main and moderator effects. Designing CRTs with adequate power to detect main and moderator effects requires accurate estimates of design parameters. This research aims to advance the literature on design parameters for power analyses, specifically focusing on empirical estimates of intraclass correlations (ICCs). The work consists of three research papers that examine the role of including the teacher level in …

Applying An Intersectional And Multi-Systemic Level Approach To Adei Institutional Assessments In Social Work Higher Education, Diana A. Melendez

Dissertations, Theses, and Capstone Projects

Institutional Anti-racism, Diversity, Equity and Inclusion (ADEI) tools and assessment models are needed to support schools of social work in demonstrating compliance with the Council on Social Work Education’s (CSWE) 2022 Educational Policy and Accreditation Standards (EPAS). This study develops a new tool and model for ADEI institutional assessment in social work higher education using a collaborative, intersectional and multi-systemic level approach. The primary aim of the project was to develop and test a new tool to gather and document experience(s) of coloniality within a social work higher education institution, from multiple stakeholders’ perspectives, as a usable model of ADEI …

Factors That Impact New York City Public High School Graduation: Finding Barriers To Education Through Data Analysis And Visualization, Kyoung Kang

Dissertations, Theses, and Capstone Projects

This capstone project is to provide visual evidence about the factors related to high school graduation rates of New York City public high school students. By analyzing the NYC public high school data, including graduation rates, school survey results, and school funding allocations, this project explores the graduation gaps among different student bodies, such as ethnic groups and school locations. The graduation disparities across the five boroughs are further examined for students with disabilities, economic difficulties, and language barriers. School-level data is used to visualize the gaps in school performances and funding allocations across the five boroughs. Lastly, the visualization …

An Empirical Study Of Student Satisfaction At Selected Private Universities In Bangladesh, Mubina Khondkar, Abureza M. Muzareba

Journal of Educational Research and Practice

Satisfying students in higher education has proven to be a difficult task, although universities are striving to meet student needs and expectations. Private universities are trying to become more student-oriented and are seeking to understand students’ perceptions of higher education facilities. This research was designed to measure the gap between students’ expectations and perceptions of various aspects of private universities in Bangladesh. It uses a SERVQUAL framework and the Gaps Model of Service Quality. The population of students at private universities in Bangladesh is represented by a sample of 412 respondents from five top-tier private universities. Findings indicate that the …

Transnational Higher Education And Quality Assurance : A Study Of 2+2 Student Learning Experience From Sino-Uk Cooperative Universities, Guoguo Ke

Lingnan Theses (Doctoral degree)

Background: The unavoidable effects of globalisation and increased competition amongst nations have fuelled students’ pursuit of international higher education. In response to this development, the adoption of transnational higher education has become a major global trend. For instance, the Chinese Ministry of Education (MoE) initiated Sino–foreign cooperative education in different forms to reform its education system whilst introducing high-quality educational resources from overseas. As one of the most popular forms, the articulation programme (2+2 programme) has attracted many students. Thus, the quality assurance of this programme warrants further exploration.

Aims and objectives of the study: The present study focuses on …

Pisa 2022. Reporting Australia’S Results. Volume Ii: Student And School Characteristics, Lisa De Bortoli, Catherine Underwood, Tim Friedman, Eveline Gebhardt

OECD Programme for International Student Assessment (PISA) Australia

The Programme for International Student Assessment (PISA) is an international comparative study of student performance directed by the Organisation for Economic Co-operation and Development (OECD). PISA measures the cumulative outcomes of education by assessing how well 15-year-olds, who have nearly completed compulsory schooling in most participating educational systems, are prepared to use their knowledge and skills in particular areas to meet real-world opportunities and challenges. In addition to the cognitive data reported on in Volume I, PISA collected a wealth of student and school contextual data through the background questionnaires. This report focuses on a variety of constructs related to …

Analyzing The Cognitive Complexity Of The Questions On The Advanced Placement Computer Science Principles Exam, Lauren Zirpoli

Seton Hall University Dissertations and Theses (ETDs)

This convergent parallel mixed methods study with qualitative and quantitative content analysis was conducted to analyze and describe the cognitive complexity of the publicly released Advanced Placement Computer Science Principles Exam questions compared to the language of higher-order thinking found in research literature. Hess’ Cognitive Rigor Matrix was used to determine the frequency and percentage of questions categorized as higher-order thinking within each cell of the matrix. The qualitative method consisted of content analysis of the language of each question analyzed, followed by deductive coding, and culminated with categorizing the depth and type of thinking required on Hess’ Cognitive Rigor …

Borderland Voices: Exploring The Educational Journey Of Transfronterizx Students, Families, And Educators For Enhanced Engagement And Empowerment, Sobeida Velazquez

Dissertations

Transfronterizx students and their families cross the U.S.–Mexico border for academic, economic, social, cultural, and linguistic reasons. Socioeconomic disparities, deportation, and work have propelled some families to live in Mexico and enroll their U.S.-born children in U.S. schools to provide more socioeconomic opportunities in the United States. Educators of transfronterizx students are uniquely tasked to work with these nontraditional students. Moreover, transfronterizx students and their families have distinct needs in U.S. schools; as such, there is a need for further research on the transfronterizx experience in the U.S. K–12 system. This qualitative narrative inquiry study aimed to understand the experiences …

Financial Literacy And Aspirations: Experimental Evidence From Eastern Uganda, Ester Agasha

Master's Theses

Financial literacy and the cultivation of aspirations are pivotal in empowering individuals and fostering economic growth, particularly within disadvantaged com munities. Although prior research has extensively analyzed the individual effects of financial education and motivational interventions on economic behaviors, there is less clarity on how the synergy between them can collectively impact the socioe conomic status of individuals in developing economies, especially women. Through randomized control experiment(RCT), this study explored the effectiveness of two interventions; 1) a financial literacy digital gaming application and, 2) an aspirations intervention (a short video documentary and a follow-up curriculum) in not only enhancing financial …

Using An Interrupted Case Study To Engage Undergraduates’ Critical Thinking Style And Enhance Content Knowledge, Kelsey Hall, Katherine Starzec

Journal on Empowering Teaching Excellence

Access the online Pressbooks version of this article here.

The interrupted case study is a structured way to engage students in active learning. Interruptions, or pauses for reflection and discussion scheduled within the case-study presentation, provide students with a chance to collaborate and engage in critical thinking. Critical thinking style, which is a measure of how one tends to think critically, provides insight into how one tackles problem solving. This article describes a pilot project that combined critical-thinking style and an interrupted case study, delivered over a two-class-period time frame, to four college courses. The project’s goals were to assess …

Navigating Epistemological Barriers For Inclusive Assessment: A Collaborative Self-Study, Yuankun Yao, Comfort M. Ateh

Education Faculty Publications

While inclusive pedagogy has been increasingly embraced in teacher education, inclusive assessment of student learning remains an area that has received limited attention. A critical friend (Ateh) from an outside institution encouraged Yao to examine how he approached inclusive assessment in a course on assessment strategies for preservice teachers. The collaborative self-study showed that his initial understanding of inclusivity was limited when he taught and practiced inclusive strategies in the assessment course. Yao’s reflections came upon moments of tensions as he navigated through barriers for conceptualizing and implementing inclusive assessment strategies. The weekly interactions with his critical friend, helped Yao …

An Evaluation Of A Cooperative Extension Internship Program, Lisa Ellis Mccormick, Karen A. Vines, Subrato Kumar Kuri

Journal of Extension

The purpose of this project was to evaluate the Virginia Cooperative Extension internship program from the perspective of student participants and their supervisors. Three focus groups were conducted with internship supervisors from the summer of 2019. Student survey data was used to identify concerns from the student perspective to inform the supervisor focus group questions. In addition to the questions, a Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis was also used to collect qualitative data within the supervisor focus groups. Findings suggest areas of strength that can be used in marketing as well as opportunities for program improvement.

Educator Perceptions Of Working With Dreambox Learning, H. Colleen Sinclair, Nathaniel Armour, Judith L. F. Rhodes, Alyssa Spears

Reports

As part of a research-practitioner partnership with East Baton Rouge Parish School System (EBRPSS), LSU Social Research & Evaluation Center, and DreamBox Learning, we wanted to evaluate educator perceptions of the implementation of a digital learning platform into K-12 math classes. Our aim was to find ways to optimize both the program and the implementation to best enable educators to facilitate student success in math, especially for students who may be struggling. To do this, our initial step was to go to the educators and simply ask them about their experiences. This report outlines the procedures and the detailed examples …