Educational Assessment, Evaluation, and Research Commons^™

Outcomes And Impacts Of A Statewide Teen Leadership Program, Meggan Callister, Shannon Cromwell

Outcomes and Impact Quarterly

The Junior Youth Conference (JYC) Teen Leadership program provides opportunities for Utah teens to plan and implement experiential workshops for 5th-6th graders throughout the state. Evaluation results indicate that teens developed leadership skills, increased self-awareness, and gained effective communication skills, thereby increasing protective factors and mitigating risky behaviors.

Improving Relationships – Hidden Gems Adventure Guides, Naomi Brower, Shannon Cromwell, Elizabeth Davis

Outcomes and Impact Quarterly

The USU Hidden Gems Adventure Guides are free online resources for families and couples. They offer one-hour activities that promote social-emotional learning. Online surveys showed improvements in family and couple outcomes in several important aspects, such as spending time together and communication skills.

Assessing The Impact Of The Snap-Ed Social Marketing Campaign, Brittney Johnson, Lacee Jimenez, Lea Palmer, Heidi Leblanc

Outcomes and Impact Quarterly

The Create Better Health (CBH) social marketing team shares practical nutrition tips, budget-friendly recipes, and resources to reinforce direct education class messages through social media platforms. An evaluation project aimed to refine Utah SNAP-Ed’s social marketing strategy. Methods included a literature review, a needs assessment survey, a strategy update, and an evaluation survey. Results showed increased engagement and motivation for healthier choices. The public value lies in reducing disparities and enhancing statewide health outcomes.

Assessing The Short-Term Outcomes Of Erase And Replace: A Series Of Health & Wellness Activities, Gabriela Murza, Christina Pay, Eva Timothy, Melanie Dabb, Cris Meier

Outcomes and Impact Quarterly

Utah State University Extension faculty addressed Utah's high rate of poor mental health with activities that improved participants’ connections, confidence, and abilities. These efforts positively impacted adults' mental health and quality of life and offered an additional resource to address self-care needs while encouraging connection and skill-building.

Imitasd: Imitation Assessment Model For Children With Autism Based On Human Pose Estimation, Hany Said, Khaled Mahar, Shaymaa E. Sorour, Ahmed Elsheshai, Ramy Shaaban, Mohamed Hesham, Mustafa Khadr, Youssef A. Mehanna, Ammar Basha, Fahima A. Maghraby

Instructional Technology and Learning Sciences Faculty Publications

Autism is a challenging brain disorder affecting children at global and national scales. Applied behavior analysis is commonly conducted as an efficient medical therapy for children. This paper focused on one paradigm of applied behavior analysis, imitation, where children mimic certain lessons to enhance children’s social behavior and play skills. This paper introduces IMITASD, a practical monitoring assessment model designed to evaluate autistic children’s behaviors efficiently. The proposed model provides an efficient solution for clinics and homes equipped with mid-specification computers attached to webcams. IMITASD automates the scoring of autistic children’s videos while they imitate a series of lessons. The …

Designing A Human-Centred Learning Analytics Dashboard In-Use, Riordan Alfredo, Vanessa Echeverria, Linxuan Zhao, Luettamae Lawrence, Jie Xiang Fan, Lixiang Yan, Xinyu Li, Zachari Swiecki, Dragan Gašević, Roberto Martinez-Maldonado

Instructional Technology and Learning Sciences Faculty Publications

Despite growing interest in applying human-centred design methods to create learning analytics (LA) systems, most efforts have concentrated on initial design phases, with limited exploration of how LA tools and practices can coevolve during the actual learning and teaching activities. This paper examines how a human-centred LA dashboard can be further refined and adapted by teachers while actively using it in a real-world scenario (i.e., design-in-use), beyond its intended design (i.e., design-for-use). We use instrumental genesis as a theoretical lens to analyze the temporary and permanent instrumentalization of design features and individual and collective instrumentation of the LA …

Outcomes Of An Educational Series For Latino Entrepreneurs In Utah, Celina Wille

Outcomes and Impact Quarterly

Latino-owned businesses represent one of the fastest-growing sectors of the U.S. economy. Critical to the success of Latino entrepreneurs are enhanced access to capital and education in best practices such as financial management and customer engagement. Utah State University (USU) Extension, in collaboration with Latino Entrepreneurs of Northern Utah (ELNU), developed and delivered a targeted educational series in fall 2021 and spring 2022. The series effectively increased participants’ knowledge and satisfaction, with a final survey showing 47% to 60% of participants planning to adopt new business practices. The initiative highlights the importance of tailored educational programs in empowering Latino business …

Nourishtank: Participant Experiences And Feedback, Palak Gupta, Heidi Leblanc, Lea Palmer

Outcomes and Impact Quarterly

NourishTank is a shark-style competition that empowers students to develop innovative, sustainable solutions for combating hunger and ensuring food security. Following the inaugural NourishTank event, participants provided valuable feedback, including an enhanced understanding of hunger issues, improved hard and soft skills, and expressed interest in volunteering. Feedback emphasized academic and personal network channels for awareness and suggestions for improvement related to program refinement and expansion of outreach efforts.

Evaluating The "What's Up Down South" Economic Summit: Transforming Feedback Into Value And Relevance, Paul A. Hill

Outcomes and Impact Quarterly

This article explores the impact of the "What's Up Down South" economic summit. Reviewing post-2023 summit changes reveals how adjustments address southern Utah's diverse business needs. Findings show greater attendee satisfaction, increased capacity, and demographic diversity, confirming the summit's role in economic development.

Assessing Community Needs And Concerns In Iron County, Meagan Johnston, Lendel Narine

Outcomes and Impact Quarterly

Utah State University Extension conducted a needs assessment to identify priority needs in Iron County. The needs assessment employed a three-phase approach involving interviews, surveys, and data analysis. Results revealed top priority issues, including youth mental health, preparing youth for successful careers, and employment opportunities. The findings will guide Extension programming in addressing community needs effectively.

Raising Awareness Of The Link Between Coal Mining And Mental Health, Kristen Koci, Jessica D. Ulrich-Schad, Courtney G. Flint, Ashley Yaugher, Amanda Mcintosh, Gabriela Murza, Aaron Hunt

Outcomes and Impact Quarterly

Mental health is a concern in Carbon and Emery Counties of Utah, particularly in its association with coal mining. An event was held to raise awareness about how the industry impacts mental health and resources to address it. Results from pre- and post-surveys show the usefulness of the event to attendees.

The Intersection Of Place And Game Design: The Case Of Two Classic African Board Games, Rebecca Y. Bayeck, Joseph M. Bayeck

Instructional Technology and Learning Sciences Faculty Publications

The concept of game design is often associated with a computational creative task or a creation with technology. Automatically linking game design to computational activity or technology may have contributed to the scarce research on the interaction between place and game design. The focus on technology may have excluded indigenous games from game design conversation in the game design literature. This article explores design in the context of two classic African board games. We examine the influence of place on the design of these games. Building on game design and place literature, the article shows the situatedness of game design …

Acknowledgments, Matthew T. Wappett Ph.D.

Developmental Disabilities Network Journal

No abstract provided.

Cognitive Reappraisal: The Bridge Between Cognitive Load And Emotion, Rebecca Brockbank, David F. Feldon

Instructional Technology and Learning Sciences Faculty Publications

Within this integrative review, cognitive load theory (CLT) is asserted as a powerful framework for conceptualizing human cognitive processes within learning. The relationship between cognition and emotion is then examined and further integrated within the scope of CLT. Emotion regulation strategies are discussed and adaptive strategies are proposed as being of particular relevance to broadening the theoretical and practical impacts of CLT. Central to the argument of this review is the use of cognitive reappraisal as a potential mitigator of cognitive load. Cognitive reappraisal involves reframing or reassessing understandings or beliefs that underlie an emotional response, which may mitigate cognitive …

Accuracy And Effectiveness Of An Orchestration Tool On Instructors' Interventions And Groups' Collaboration, Luettamae Lawrence, Emma Mercier, Taylor Tucker Parks, Nigel Bosch, Luc Paquette

Instructional Technology and Learning Sciences Faculty Publications

This paper presents the development of a novel orchestration tool that predicts collaborative problem-solving (CPS) behaviors of undergraduate engineering groups and investigates the use of that tool by instructors. We explore the impact of receiving real-time, machine-learning, model-based prompts on 1) instructors' orchestration strategies, which are strategies instructors use to manage and facilitate collaborative activities, and 2) groups' participation, including how groups are engaged in CPS activities. The orchestration tool is a dashboard that notifies instructors of–and advises them on–monitoring and intervening with groups who may need collaborative support and guidance. We describe the accuracy of the models in predicting …

Evaluating 45 Years Of Collaborative Range Livestock Education, Melanie Heaton, Andrew Brischke, Ben Scow

Outcomes and Impact Quarterly

The Arizona/Utah Range Livestock Workshop & Tour builds cooperation and understanding between public land grazing permittees and federal agency personnel through science-based education. The workshop provides information on livestock production practices and addresses controversial grazing issues in a non-confrontational environment. Topics range from animal handling, vaccination, and health to solar development, partnering with public land agencies, and recreation effects.

Impact Of The 2023 Central Utah Grazing Expo, Trenton Wilde, Eric Thacker, Ryan Larsen, Chad Page, Burdette Barker, Mark Nelson, Matt Palmer, Steven Price, Jacob Hadfield, Kalen Taylor, Ethan Gilliam

Outcomes and Impact Quarterly

A collaboration between Extension Agriculture and Natural Resource specialists and county faculty responded to a need to educate public land grazing permit holders, public land management agency personnel, and public members on regulations, management, economics, and social values associated with public land grazing permits.

The Utah Urban And Small Farms Conference: A Model For Small Acreage Producer Extension Programming, Katie Wagner, Ruby Ward, Kynda R. Curtis, Sheriden M. Hansen, Melanie Stock

Outcomes and Impact Quarterly

The Utah Urban and Small Farms Conference (USFC) provides outreach to small acreage and urban producers facing urbanization and environmental challenges and offers marketing and profitability opportunities. The annual event attracts new and existing farmers, stakeholders, and representatives. The USFC model may assist communities facing similar challenges, helping producers navigate obstacles and opportunities through localized information sharing.

Computational Models As Tools For Supporting Responsive Teaching, Hillary Swanson, Luettamae Lawrence, Jared Arnell, Bonni Jones, Bruce Sherin, Uri Wilensky

Instructional Technology and Learning Sciences Faculty Publications

It is widely agreed that science instruction should help students build new knowledge on the foundation of their prior knowledge. Responsive teaching refers to a family of teaching strategies that pursue and build on student ideas. We introduce a particular approach to responsive teaching and examine how it can be supported by the use of computational models. We analyze an 8^th grade science teacher's facilitation of a class discussion near the end of a lesson on sound. We present a moment-by-moment characterization of her responsive teaching moves, highlighting the ways she used a computational model to help students articulate …

A Process Evaluation Of Utah’S Food Security Council, Palak Gupta, Heidi Leblanc

Outcomes and Impact Quarterly

Utah’s Food Security Council (FSC) brings together diverse stakeholders to address food security across the state. As FSC concluded its inaugural year of work, a process evaluation assessed its progress. Findings will inform FSC’s future strategic improvements, partnerships, and community outreach activities.

A National Perspective Of Mental Health Awareness Among Agricultural Workers, Anna Gilliam, Ethan Gilliam

Outcomes and Impact Quarterly

The knowledge and perceptions of mental health resources for agricultural workers were explored at the 2024 American Farm Bureau Federation (AFBF) National Conference. Data were gathered via a voluntary survey conducted at the event. With 137 completed surveys, the study revealed that the participants desired more information regarding agricultural mental health resources and revealed their preferences for receiving this information. These findings will lead USU Extension to create impactful educational resources to better address mental health awareness and education needs in agricultural workers.

Incorporating Design Based Implementation Research With A Randomized Controlled Trial To Develop And Evaluate The Efficacy Of Playful Rational Number Learning, Kreshnik N. Begolli, Vanessa N. Bermudez, Luettamae Lawrence, Lourdes M. Acevedo-Farag, Sabrina V. Valdez, Evelyn Santana, Daniela Alvarez-Vargas, June Ahn, Drew Bailey, Katherine Rhodes, Lindsey E. Richland, Andres S. Bustamante

Instructional Technology and Learning Sciences Faculty Publications

We combine design-based implementation research with a pre-registered RCT to address a long-standing challenge in psychological science: How to use psychological principles to address real-world problems while designing and implementing interventions in the field. We posit this as a design methodology for optimizing the translation between psychological science and real-world applications. We tested the efficacy of an extensively co-designed version of a game-based rational number intervention, Fraction Ball, versus "business-as-usual" math instruction and physical education in a sample of 4th/5th grade Latine students (N=360). Insights from nine co-design sessions with 20 teachers informed revisions and additions to a …

Meta-Theoretic Competence For Computational Agent-Based Modeling, Hillary Swanson, Uri Wilensky

Instructional Technology and Learning Sciences Faculty Publications

In the U.S. context, science standards encourage educators to engage students in modeling practices, including computational modeling. While much work has investigated the productivity of computational modeling with respect to students' development of scientific content knowledge, less work has focused on students' development of knowledge and skills for participation in computational modeling practices. A first step in understanding how these practices develop is examining students' activity in the context of computational modeling environments with attention to the productive moves they make. These moves can provide insight into the knowledge they bring to their learning, which may be foundational to the …

Design Forms And Games, Hillary Swanson, Ha Nguyen, Caroline Lavoie, Trilby Nelson

Instructional Technology and Learning Sciences Faculty Publications

This paper introduces a theoretical framework for characterizing design processes as design games. Based on Collins and Ferguson's epistemic forms and games, the framework characterizes design processes as games played by designers to fill out design forms, which are general templates underlying specific design artifacts. We introduce our framework through games involved in human-centered design, describing for each the entry conditions, target form, constraints, and moves. We demonstrate how the framework can be used to characterize design games played by experts in museum exhibit design and urban design. Through similar analyses, the framework …

What Happened To The Geometry? Examining Spatial And Mathematical Concepts In Computational Toys And Kits For Young Children, Jody Clarke-Midura, Jessica F. Shumway, Anahita Ashineh, Alireza Zandi, Whitney Clawson

Instructional Technology and Learning Sciences Faculty Publications

This paper explores the design of computational toys and kits for young children (ages 4-7) as tools for learning integrated mathematical, spatial, and computational thinking concepts. Specifically, we examine how the design features of the toys and kits represent the concepts of rotation on a point and spatial orientation of the agent. We examine toys and kits sold commercially, developed through research, and used in early childhood classrooms. Our findings indicate that the mathematical and spatial concepts are overlooked in some designs. Prior research examined toys for their affordances related to computational thinking, the present study contributes to understanding of …

The Role Of Generative Ai In Qualitative Research: Gpt-4'S Contributions To A Grounded Theory Analysis, Ravi Sinha, Idris Solola, Ha Nguyen, Hillary Swanson, Luettamae Lawrence

Instructional Technology and Learning Sciences Faculty Publications

We present reflections on our experience using a generative AI model in qualitative research, to illuminate the AI's contributions to our analytic process. Our analytic focus was a segment of classroom transcript, which captured a teacher introducing scientific theory-building practices to middle school students. We used a grounded theory approach to produce a fine-grained characterization of the teacher's talk moves during the lesson implementation. Our eventual goal is to build a more nuanced conceptualization of responsive teaching in the context of theory-building activities. We involved GPT-4 during the initial exploratory and later focused coding stages. For our analysis of GPT-4's …

Theory-Based Science Communication At Comic Cons, Lisa Lundgren, Emily Slater, Man Zhang, Kadie Kunz, Gabriel-Philip Santos

Instructional Technology and Learning Sciences Faculty Publications

The majority of research concerning science communication happens in spaces where people already have a vested interest in or knowledge of science, such as museums and science cafes. Thus, there is a gap in understanding what theory-based science communication looks like in non-science-centered spaces. This qualitative research study, which featured cosplaying science communicators at comic conventions, offers insights into what science communication practices occur within everyday spaces. We conducted observations of and semi-structured interviews with 15 cosplaying scientists, examining how they utilized effective framing and narrative structuring when communicating science at comic cons in cosplay. Across 700 coded utterances, cosplaying …

Who Gets To Decolonize: A Reflection On The Importance Of Positionality In The Decolonization Of Digital Learning Spaces And Learning Design, Rebecca Y. Bayeck, Tutaleni Iita Asino

Instructional Technology and Learning Sciences Faculty Publications

Decolonization as a term and concept gained prominence in the 20th century as new nations emerged from the cessation of the global colonial project which primarily saw European countries occupying different parts of the world. In modern times, interests in decolonization have taken different forms, particularly in learning design and digital learning spaces. In the digital space, researchers mainly suggest introducing indigenous perspectives and cultural lenses in the design of digital learning environments and the design of emerging technologies that embed indigenous perspectives (Nede, 2023). This reflexive essay discusses the importance of positionality in digital learning spaces and learning/instructional design. …

Understanding Non-Designers' Practices And Processes In A Human-Centered Design Course, Luettamae Lawrence, Saadeddine Shehab, Mike Tissenbaum

Instructional Technology and Learning Sciences Faculty Publications

Human-centered design (HCD) has been identified in the literature as a useful problem-solving approach for learners. However, learning and applying HCD poses several challenges for students who are unfamiliar with this form of learning. In this paper, we analyse how a novice design team worked on a HCD class project to examine how non-designers learn about and integrate HCD practices into their project. We introduce the HCD taxonomy to define the processes and practices that students engage in. The team’s design work is triangulated across multiple data sources and revealed three challenges for engaging non-designers in HCD: 1) the need …

Using An Interrupted Case Study To Engage Undergraduates’ Critical Thinking Style And Enhance Content Knowledge, Kelsey Hall, Katherine Starzec

Journal on Empowering Teaching Excellence

Access the online Pressbooks version of this article here.

The interrupted case study is a structured way to engage students in active learning. Interruptions, or pauses for reflection and discussion scheduled within the case-study presentation, provide students with a chance to collaborate and engage in critical thinking. Critical thinking style, which is a measure of how one tends to think critically, provides insight into how one tackles problem solving. This article describes a pilot project that combined critical-thinking style and an interrupted case study, delivered over a two-class-period time frame, to four college courses. The project’s goals were to assess …