Educational Assessment, Evaluation, and Research Commons^™

21st Century Community Learning Centers: Program Evaluation Evidence From Texas, Rosilyn Jackson

Electronic Theses and Dissertations

Texas afterschool programs strive to enhance academic performance and meet students' needs, yet gaps in program quality persist. This study evaluated 21st Century Community Learning Centers’ performance in reading, math, GPA, attendance, in-school suspension, and engagement, both within these categories and against program goals and demographics. Theoretical frameworks used in this study include the logic model, social justice theory, and cultural relevance theory. This quantitative study utilized a nonexperimental and secondary data research design which utilized five statistical tests: the Pearson Product-Moment Correlation, One-Way repeated ANOVA, Spearman Correlation, the Independent Samples t-Test, and the Mann-Whitney U Test. Results showed that …

A Qualitative Study Of Iranian Efl Teachers’ Perceptions Of A Digital Diversified Education, Seyed Mohammad Reza Marvasti Zadeh, Azizollah Dabaghi Varnosfadrani Advarnosfadrani, Mohammad Amiryousefi

The Qualitative Report

Because Iran is a country of learners of diverse languages, socio-economic classes, genders, and ages, digital diversified education seems appropriate to address needs in English language education. However, there is a notable gap in research on teachers’ perceptions about this type of education. To fill this gap, this study aimed to investigate the perspectives of Iranian teachers of English as a foreign language (EFL) regarding a diverse digital education. Data were collected via an open-ended survey with 30 EFL teachers, both male and female, who were selected through convenience sampling. Analysis revealed several key themes, including the utilization of diverse …

Exploring Students' Computer-Supported Collaborative Argumentation With Socio-Scientific Issues, Wenli Chen, Yiting Han, Jesmine Tan, Aileen Siew Cheng Chai, Qianru Lyu, Lyna

Research Collection Lee Kong Chian School Of Business

Background: This study examined the effect of computer-supported collaborative argumentation (CSCA) on secondary school students' understanding of socio-scientific issues (SSI). Engaging students in collaborative argumentation is known to help with deepening their understanding of SSI. Methods: In this study, a mixed-method design is used to investigate 84 students' collaborative argumentation processes and outcomes. The statistical analysis, epistemic network analysis and qualitative uptake analysis results showed that CSCA was effective in supporting secondary school students' evidence-based argumentation skills on SSI. Findings and Conclusion: Several cases were presented to show how students engaged in CSCA to explore meaningful learning opportunities and how …

Scholar Perspectives On The Impact Of A Scientific Community Program For Neurodivergent Undergraduate Stem Scholars, Dylan Sullivan, Fernando Zavala, Derek Hidalgo, Jacob Stolle, Rebecca Matte, Christin B. Monroe

Journal of Science Education for Students with Disabilities

Despite the adaptive strengths and unique problem-solving skills demonstrated by neurodivergent (ND) individuals, they remain underrepresented in Science, Technology, Engineering and Mathematics (STEM) fields. High unemployment rates among individuals with disabilities emphasize the need for addressing barriers to entry and persistence in the workforce. This study introduces a program designed to enhance opportunities for neurodivergent STEM scholars with financial needs, supported by the National Science Foundation (NSF). The program involves: 1) a weekly cohort course to engage in professional development, 2) use of the Birkman Method® survey to help scholars identify and communicate strengths, fostering self-awareness and growth, and 3) …

Unraveling The Impact Of Cognitive Test Anxiety On Academic Performance Among Adolescents: The Case Of Minority And Majority Communities, Zulfiqar Ullah Siddiqui, Wael M. F. Abuhassan

An-Najah University Journal for Research - B (Humanities)

Background: Within the Indian society and its many communities, adolescents exhibit a wide range of individual differences, including cognitive abilities, learning styles, and personality traits. These differences can influence how students perceive and cope with test anxiety and related stress. Objective: The primary goal of this research study was to investigate the influence of cognitive test anxiety on academic performance in adolescents from both minority (Muslims) and majority (Hindus) communities in India. Method: Based on descriptive-inferential research design, a sample of 450 students was selected, with 223 students representing the majority community and the remaining 227 students representing the minority …

Pisa 2022. Assessing Creative Thinking For A Better Future, Lisa De Bortoli, Catherine Underwood

OECD Programme for International Student Assessment (PISA) Australia

Creative thinking was assessed as an innovative domain for the first time in PISA 2022. This report presents the creative thinking results for Australia as a whole, for the Australian states and territories and for the other groups in PISA 2022. It is one of two thematic reports providing additional high-level analysis of Australia's 2022 PISA performance. Differences in results are only reported if tests of statistical significance showed that these were likely to be real differences; that is, differences were unlikely to be caused by chance.

Designing A Human-Centred Learning Analytics Dashboard In-Use, Riordan Alfredo, Vanessa Echeverria, Linxuan Zhao, Luettamae Lawrence, Jie Xiang Fan, Lixiang Yan, Xinyu Li, Zachari Swiecki, Dragan Gašević, Roberto Martinez-Maldonado

Instructional Technology and Learning Sciences Faculty Publications

Despite growing interest in applying human-centred design methods to create learning analytics (LA) systems, most efforts have concentrated on initial design phases, with limited exploration of how LA tools and practices can coevolve during the actual learning and teaching activities. This paper examines how a human-centred LA dashboard can be further refined and adapted by teachers while actively using it in a real-world scenario (i.e., design-in-use), beyond its intended design (i.e., design-for-use). We use instrumental genesis as a theoretical lens to analyze the temporary and permanent instrumentalization of design features and individual and collective instrumentation of the LA …

The Value Of Cosma Outcomes: Student Learning Outcomes, Transferable Skills And Taxonomy Levels, Coral Bender, Lindsey Maxwell

The COSMA Journal

Using a content analysis of all published Student Learning Outcomes (SLOs) of Commission on Sport Management Accreditation (COSMA) accredited programs, the current study analyzes SLOs to assess the most frequently used measurable verbs (utilizing a revised version of Blooms’ Taxonomy) and transferable skills (utilizing the definitions provided by AAC&U’s VALUE Outcomes and their corresponding rubrics) emphasized by accredited programs. An analysis of the most common Taxonomy Levels is presented at the aggregate level as well as a comparison between undergraduate and graduate education to provide a case study of SLO measurable verb alignment across accredited sport management programs. Further, content …

What In The Assessment Are We Doing? A Snapshot Of Student Learning Assessment In Health Science Libraries, Danielle Westmark, Teresa L. Hartman

Posters and Presentations: Leon S. McGoogan Health Sciences Library

Objective: When it comes to education and instruction, there’s no clear answer to what health science librarians/information professionals are doing in student learning assessment and if the methodologies they use are helpful and successful in receiving responses. This project looks to answer two questions: do health science librarians assess student learning in the context of health-related information literacy, and what strategies do health science librarians employ to encourage students to complete learning assessments?

Methods: A twelve-question survey was developed using the Springshare LibWizard tool. It was sent out to library listservs: MCMLA, ICON, MEDLIB-L, AAHSL (Deans/Directors), ACRL, Health …

The Intersection Of Place And Game Design: The Case Of Two Classic African Board Games, Rebecca Y. Bayeck, Joseph M. Bayeck

Instructional Technology and Learning Sciences Faculty Publications

The concept of game design is often associated with a computational creative task or a creation with technology. Automatically linking game design to computational activity or technology may have contributed to the scarce research on the interaction between place and game design. The focus on technology may have excluded indigenous games from game design conversation in the game design literature. This article explores design in the context of two classic African board games. We examine the influence of place on the design of these games. Building on game design and place literature, the article shows the situatedness of game design …

Sport Social Hub: Using The Power Of Collaboration To Develop Hands-On Social Media Competencies, Alisa Agozzino, Brendan O'Hallarn, Peyton Stensland, James Strode

The COSMA Journal

This paper introduces Sport Social Hub, a collaborative, multi-institutional social media exchange inspired by the Commission on Public Relations Education's (CPRE) emphasis on key knowledge, skills, and abilities (KSAs) in public relations education. Sport Social Hub engaged students from four universities in topical, sports-themed assignments. These assignments involved creating memes, analyzing historical and contemporary sports photographs, profiling industry professionals with infographics, and developing engagement contests. The paper outlines how Sport Social Hub addresses CPRE's recommended KSAs within social media, collaboration, and creative thinking. Utilizing social media platforms as pedagogical tools, Sport Social Hub aimed to enhance students' social media competencies, …

Improving Educator Readiness To Support Student Mental Health, Anna Dabrowski, Michelle Hsien, Tamara Van Der Zant, Syeda Kashfee Ahmed, Amy Berry, Maya Conway, Yung Nietschke

Wellbeing

This project used a mixed-method approach to understand educator readiness to support student mental health across early childhood, school based, and tertiary settings. The study features educators and relevant support staff working with children, adolescents, and adults in formal learning settings. Relevant experts such as mental health care workers, youth wellbeing support staff, counsellors, psychologists, and other professionals involved in education settings were also invited to take part in the study. Three research questions guided the focus of the study: How does educator anxiety and worry impact on their ability to support student mental health? To what extent are educators …

Cognitive Reappraisal: The Bridge Between Cognitive Load And Emotion, Rebecca Brockbank, David F. Feldon

Instructional Technology and Learning Sciences Faculty Publications

Within this integrative review, cognitive load theory (CLT) is asserted as a powerful framework for conceptualizing human cognitive processes within learning. The relationship between cognition and emotion is then examined and further integrated within the scope of CLT. Emotion regulation strategies are discussed and adaptive strategies are proposed as being of particular relevance to broadening the theoretical and practical impacts of CLT. Central to the argument of this review is the use of cognitive reappraisal as a potential mitigator of cognitive load. Cognitive reappraisal involves reframing or reassessing understandings or beliefs that underlie an emotional response, which may mitigate cognitive …

Accuracy And Effectiveness Of An Orchestration Tool On Instructors' Interventions And Groups' Collaboration, Luettamae Lawrence, Emma Mercier, Taylor Tucker Parks, Nigel Bosch, Luc Paquette

Instructional Technology and Learning Sciences Faculty Publications

This paper presents the development of a novel orchestration tool that predicts collaborative problem-solving (CPS) behaviors of undergraduate engineering groups and investigates the use of that tool by instructors. We explore the impact of receiving real-time, machine-learning, model-based prompts on 1) instructors' orchestration strategies, which are strategies instructors use to manage and facilitate collaborative activities, and 2) groups' participation, including how groups are engaged in CPS activities. The orchestration tool is a dashboard that notifies instructors of–and advises them on–monitoring and intervening with groups who may need collaborative support and guidance. We describe the accuracy of the models in predicting …

Self-Directed Learning In A Qualitative Research Methods Course For Doctoral Students, Amy L. Damrow, Krittika Grau, Brandon Pettry, Karen Plaster, Adepeju Prince

The Qualitative Report

This paper describes (1) the reasoning behind a self-directed learning assignment in an advanced qualitative research methods course, and (2) doctoral students’ experiences with that assignment. Drawing on instructor and student perspectives, a professor and four doctoral students collaborated to reflect and report on the process and product of one assignment within a course. The doctoral students approached the assignment in unique ways: delving into one specific research methodology, comparing multiple research designs, and exploring current uses of qualitative research at a research conference. The piece contributes to the body of literature about students’ and instructors’ experiences in graduate qualitative …

Link Virtual Forum On Authentic Assessment For Early Childhood Intervention: Interdisciplinary & International Consensus, Stefano J. Bagnato, Marisa Macy, Nora Smith, Carmen Dionne, Mary Beth Bruder, Maria Londono, Jackie Robinson Brock, Antonio Fevola, Tracy Larson

Perspectives on Early Childhood Psychology and Education

The LINK Virtual Forum on Authentic Assessment for Early Childhood Intervention: Interdisciplinary & International Consensus was orchestrated through a series of virtual focus-groups during the pandemic in 2021-2022. This new strategy for conducting a more robust national/international consumer social validity study gathered practice-based evidence from the field using national and international expert panel focus groups and an expert panel forum to survey Authentic Assessment experts and users. Uniquely, these deliberations relied upon the Nominal Group Technique (NGT) among participants to reach consensus decisions and Qualtrics survey results. The decision-making process addressed types of assessment measures that address early childhood …

Hard Or Soft: An Exploration Of What Skills A Sport Management Student Is Lacking, Ashley Ryder, Jillian Mcniff Villemaire, Taylor Behl

The COSMA Journal

An important measure of success of sport management programs in higher education is how prepared students are for their professional careers. Programs often include various experiential learning opportunities such as internships in their curriculum to improve student employability. However, there is still concern that students may not be prepared for success in their internships and in the field. The purpose of this study was to identify which skills sport management student interns needed the most improvement, according to their intern supervisor. Using a single institution, this study analyzed intern supervisor’s questionnaire responses from 2017-2022 at a small, private college in …

Computational Models As Tools For Supporting Responsive Teaching, Hillary Swanson, Luettamae Lawrence, Jared Arnell, Bonni Jones, Bruce Sherin, Uri Wilensky

Instructional Technology and Learning Sciences Faculty Publications

It is widely agreed that science instruction should help students build new knowledge on the foundation of their prior knowledge. Responsive teaching refers to a family of teaching strategies that pursue and build on student ideas. We introduce a particular approach to responsive teaching and examine how it can be supported by the use of computational models. We analyze an 8^th grade science teacher's facilitation of a class discussion near the end of a lesson on sound. We present a moment-by-moment characterization of her responsive teaching moves, highlighting the ways she used a computational model to help students articulate …

Meta-Theoretic Competence For Computational Agent-Based Modeling, Hillary Swanson, Uri Wilensky

Instructional Technology and Learning Sciences Faculty Publications

In the U.S. context, science standards encourage educators to engage students in modeling practices, including computational modeling. While much work has investigated the productivity of computational modeling with respect to students' development of scientific content knowledge, less work has focused on students' development of knowledge and skills for participation in computational modeling practices. A first step in understanding how these practices develop is examining students' activity in the context of computational modeling environments with attention to the productive moves they make. These moves can provide insight into the knowledge they bring to their learning, which may be foundational to the …

Design Forms And Games, Hillary Swanson, Ha Nguyen, Caroline Lavoie, Trilby Nelson

Instructional Technology and Learning Sciences Faculty Publications

This paper introduces a theoretical framework for characterizing design processes as design games. Based on Collins and Ferguson's epistemic forms and games, the framework characterizes design processes as games played by designers to fill out design forms, which are general templates underlying specific design artifacts. We introduce our framework through games involved in human-centered design, describing for each the entry conditions, target form, constraints, and moves. We demonstrate how the framework can be used to characterize design games played by experts in museum exhibit design and urban design. Through similar analyses, the framework …

The Role Of Generative Ai In Qualitative Research: Gpt-4'S Contributions To A Grounded Theory Analysis, Ravi Sinha, Idris Solola, Ha Nguyen, Hillary Swanson, Luettamae Lawrence

Instructional Technology and Learning Sciences Faculty Publications

We present reflections on our experience using a generative AI model in qualitative research, to illuminate the AI's contributions to our analytic process. Our analytic focus was a segment of classroom transcript, which captured a teacher introducing scientific theory-building practices to middle school students. We used a grounded theory approach to produce a fine-grained characterization of the teacher's talk moves during the lesson implementation. Our eventual goal is to build a more nuanced conceptualization of responsive teaching in the context of theory-building activities. We involved GPT-4 during the initial exploratory and later focused coding stages. For our analysis of GPT-4's …

What Happened To The Geometry? Examining Spatial And Mathematical Concepts In Computational Toys And Kits For Young Children, Jody Clarke-Midura, Jessica F. Shumway, Anahita Ashineh, Alireza Zandi, Whitney Clawson

Instructional Technology and Learning Sciences Faculty Publications

This paper explores the design of computational toys and kits for young children (ages 4-7) as tools for learning integrated mathematical, spatial, and computational thinking concepts. Specifically, we examine how the design features of the toys and kits represent the concepts of rotation on a point and spatial orientation of the agent. We examine toys and kits sold commercially, developed through research, and used in early childhood classrooms. Our findings indicate that the mathematical and spatial concepts are overlooked in some designs. Prior research examined toys for their affordances related to computational thinking, the present study contributes to understanding of …

Redefining Science Education Paradigm—By Course Habitable Earth, Nianzhi Jiao, Sanzhong Li, Chen Hu, Zhonghe Zhou, Degan Shu, Shucheng Xie, Jianping Huang, Fahu Chen, Lin Ding, Tong Zhu, Yigang Xu, Yonglong Lyu, Zhimin Jian, Pinxian Wang

Bulletin of Chinese Academy of Sciences (Chinese Version)

In a rapidly evolving world defined by unprecedented challenges, the integration of education, technology, and talent development has emerged as a national strategic imperative. Cultivating interdisciplinary and innovative scientific talents is not only a demand of the era but also a cornerstone for transforming “population dividends” into “talent dividends”. Against this backdrop, the rapid advancement of artificial intelligence (AI) offers both challenges and opportunities for redefining traditional education models. This study explores the pedagogical practices of the Habitable Earth general science education course, positioning it as a case study to demonstrate how science education can synergize technological empowerment with talent …

Theory-Based Science Communication At Comic Cons, Lisa Lundgren, Emily Slater, Man Zhang, Kadie Kunz, Gabriel-Philip Santos

Instructional Technology and Learning Sciences Faculty Publications

The majority of research concerning science communication happens in spaces where people already have a vested interest in or knowledge of science, such as museums and science cafes. Thus, there is a gap in understanding what theory-based science communication looks like in non-science-centered spaces. This qualitative research study, which featured cosplaying science communicators at comic conventions, offers insights into what science communication practices occur within everyday spaces. We conducted observations of and semi-structured interviews with 15 cosplaying scientists, examining how they utilized effective framing and narrative structuring when communicating science at comic cons in cosplay. Across 700 coded utterances, cosplaying …

The Impacts Of The Higher Education Expansion On Education Quality In China : Evidence From Shenzhen University, Yousong Lun

Lingnan Theses (Doctoral degree)

Within the expansion of higher education in China, education quality should be considered into the process of reform and establishment, which is highly related to the resources of faculty and performances of management. It is crucial that universities sustainably conduct diverse academic research and meet the needs of community. Faced with the policy reform and opening up, China has more opportunities on cooperating with oversea education institutions and taking in overseas management system. This research explores the general quality assessment on higher education and the solution to balance the needs of domestic society and the growth of international education competitiveness, …

Who Gets To Decolonize: A Reflection On The Importance Of Positionality In The Decolonization Of Digital Learning Spaces And Learning Design, Rebecca Y. Bayeck, Tutaleni Iita Asino

Instructional Technology and Learning Sciences Faculty Publications

Decolonization as a term and concept gained prominence in the 20th century as new nations emerged from the cessation of the global colonial project which primarily saw European countries occupying different parts of the world. In modern times, interests in decolonization have taken different forms, particularly in learning design and digital learning spaces. In the digital space, researchers mainly suggest introducing indigenous perspectives and cultural lenses in the design of digital learning environments and the design of emerging technologies that embed indigenous perspectives (Nede, 2023). This reflexive essay discusses the importance of positionality in digital learning spaces and learning/instructional design. …

Understanding Non-Designers' Practices And Processes In A Human-Centered Design Course, Luettamae Lawrence, Saadeddine Shehab, Mike Tissenbaum

Instructional Technology and Learning Sciences Faculty Publications

Human-centered design (HCD) has been identified in the literature as a useful problem-solving approach for learners. However, learning and applying HCD poses several challenges for students who are unfamiliar with this form of learning. In this paper, we analyse how a novice design team worked on a HCD class project to examine how non-designers learn about and integrate HCD practices into their project. We introduce the HCD taxonomy to define the processes and practices that students engage in. The team’s design work is triangulated across multiple data sources and revealed three challenges for engaging non-designers in HCD: 1) the need …

Applying An Intersectional And Multi-Systemic Level Approach To Adei Institutional Assessments In Social Work Higher Education, Diana A. Melendez

Dissertations, Theses, and Capstone Projects

Institutional Anti-racism, Diversity, Equity and Inclusion (ADEI) tools and assessment models are needed to support schools of social work in demonstrating compliance with the Council on Social Work Education’s (CSWE) 2022 Educational Policy and Accreditation Standards (EPAS). This study develops a new tool and model for ADEI institutional assessment in social work higher education using a collaborative, intersectional and multi-systemic level approach. The primary aim of the project was to develop and test a new tool to gather and document experience(s) of coloniality within a social work higher education institution, from multiple stakeholders’ perspectives, as a usable model of ADEI …

Borderland Voices: Exploring The Educational Journey Of Transfronterizx Students, Families, And Educators For Enhanced Engagement And Empowerment, Sobeida Velazquez

Dissertations

Transfronterizx students and their families cross the U.S.–Mexico border for academic, economic, social, cultural, and linguistic reasons. Socioeconomic disparities, deportation, and work have propelled some families to live in Mexico and enroll their U.S.-born children in U.S. schools to provide more socioeconomic opportunities in the United States. Educators of transfronterizx students are uniquely tasked to work with these nontraditional students. Moreover, transfronterizx students and their families have distinct needs in U.S. schools; as such, there is a need for further research on the transfronterizx experience in the U.S. K–12 system. This qualitative narrative inquiry study aimed to understand the experiences …

Financial Literacy And Aspirations: Experimental Evidence From Eastern Uganda, Ester Agasha

Master's Theses

Financial literacy and the cultivation of aspirations are pivotal in empowering individuals and fostering economic growth, particularly within disadvantaged com munities. Although prior research has extensively analyzed the individual effects of financial education and motivational interventions on economic behaviors, there is less clarity on how the synergy between them can collectively impact the socioe conomic status of individuals in developing economies, especially women. Through randomized control experiment(RCT), this study explored the effectiveness of two interventions; 1) a financial literacy digital gaming application and, 2) an aspirations intervention (a short video documentary and a follow-up curriculum) in not only enhancing financial …