Education Commons^™

How Teachers Conceptualise Shared Control With An Ai Co-Orchestration Tool: A Multiyear Teacher-Centred Design Process, Luettamae Lawrence, Vanessa Echeverria, Kexin Yang, Vincent Aleven, Nikol Rummel

Instructional Technology and Learning Sciences Faculty Publications

Artificial intelligence (AI) can enhance teachers' capabilities by sharing control over different parts of learning activities. This is especially true for complex learning activities, such as dynamic learning transitions where students move between individual and collaborative learning in un-planned ways, as the need arises. Yet, few initiatives have emerged considering how shared responsibility between teachers and AI can support learning and how teachers' voices might be included to inform design decisions. The goal of our article is twofold. First, we describe a secondary analysis of our co-design process comprising six design methods to understand how teachers conceptualise sharing control with …

How Co-Designing Computational Modeling Activities Helped Teachers Implement Responsive Teaching Strategies, Hillary Swanson, Luettamae Lawrence, Jared Arnell, Allisia Dawkins, Bonni Jones, Bruce Sherin, Uri Wilensky

Instructional Technology and Learning Sciences Faculty Publications

In recent years, science education has shifted focus, from content to practice. This is reflected in the NGSS, which advocate learning science concepts through engagement in science and engineering practices. Theory building is a central activity of science and computational modeling is a key practice through which contemporary scientists construct theory. In this paper, we discuss an 8th grade science teacher's implementation of a computational modeling lesson. The teacher had co-designed the computational modeling microworld and lesson with the research team over the preceding summers. We investigate the teacher's activity during a whole-class discussion near the end of the lesson, …

Understanding Computational Thinking In The Gameplay Of The African Songo Board Game, Rebecca Y. Bayeck

Instructional Technology and Learning Sciences Faculty Publications

Computational thinking is a necessary skill for the 21st century. While previously examined in computer-rich settings, researchers are increasingly studying computational thinking in unplugged environments such as board games. Focusing on the African board game Songo, this study shows that computational thinking practices are embedded in Songo board gameplay and interact with the cultural context. The study also reveals a computing practice peculiar to Songo gameplay, songoputation. This paper suggests that researchers can benefit from exploring computational thinking and computing practices beyond board games in western contexts.

Supporting And Sustaining Equitable Steam Activities In High School Classrooms: Understanding Computer Science Teachers’ Needs And Practices When Implementing An E-Textiles Curriculum To Forge Connections Across Communities, Deborah A. Fields, Yasmin B. Kafai

Instructional Technology and Learning Sciences Faculty Publications

While the last two decades have seen an increased interest in STEAM (science, technology, engineering, arts, and mathematics) in K-12 schools, few efforts have focused on the teachers and teaching practices necessary to support these interventions. Even fewer have considered the important work that teachers carry out not just inside classrooms but beyond the classroom walls to sustain such STEAM implementation efforts, from interacting with administrators to recruiting students and persuading parents about the importance of arts and computer science. In order to understand teachers’ needs and practices regarding STEAM implementation, in this paper, we focus on eight experienced computer …

Is Microethnography An Ethnographic Case Study? And/Or A Mini-Ethnographic Case Study? An Analysis Of The Literature, Rebecca Y. Bayeck

Instructional Technology and Learning Sciences Faculty Publications

Selecting the research approach that addresses the research question is often challenging for novice researchers. However, getting a better understanding of the research approaches available in the field, is likely to help novice researchers identify and choose the research approach that fits their situation. In this paper, we discuss microethnography, ethnographic case study, and mini-ethnography case study in order to show that these approaches may have similarities but are different. The author hopes that this discussion will help researchers get a better understanding of these approaches and dissipate the confusion that may exist.

Student Behavior Ratings And Response To Tier 1 Reading Intervention: Which Students Do Not Benefit?, Wilhelmina Van Dijk, Christopher Schatschneider, Stephanie Al Otaiba, Sara A. Hart

Special Education and Rehabilitation Counseling Faculty Publications

Core reading instruction and interventions have differential effects based on student characteristics such as cognitive ability and pre-intervention skill level. Evidence for differential effect based on affective characteristics is scant and ambiguous; however, students with problem behavior are more often non-responsive to core reading instruction and intensive reading interventions. In this study, we estimated the range of students' behavior ratings in which a core reading instruction intervention was effective using a data set including 3,024 students in K-3. Data came from seven independent studies evaluating the individualized Student Instruction (ISI) Tier 1 reading intervention and were pooled using integrative data …

The Predictive Validity Of The Gre Across Graduate Outcomes: A Meta-Analysis Of Trends Over Time, David F. Feldon, Kaylee Litson, Brinleigh Cahoon, Zhang Feng, Andrew Walker, Colby Tofel-Grehl

Instructional Technology and Learning Sciences Faculty Publications

This meta-analysis assesses the predictive validity of the Graduate Record Examination (GRE) across outcome variables, including grade point average, for graduate students. In addition to aggregate effects, this paper also assessed changes in observed effects over time as related to increasing diversity in the graduate student population and as a function of gender and racial/ethnic composition of study samples. Framed using a lens of critical whiteness, this analysis examined n = 1,744 individual effects across k = 208 studies. Overall, 61.6% of reported effects were nonsignificant (i.e. no predictive value of GRE scores on student outcomes). Further, the magnitude of …

The Impact Of Covid-19 On Students' Academic Performance: The Case Of The University Of Ghana Business School, Joseph Ofori Acheampong

Political Science Student Research

The unruly consequences of the COVID-19 outbreak have hamstrung higher education institutions. This study aims to investigate the perceived impact of COVID-19 on student learning and academic performance through a qualitative approach. In-depth semi-structured interviews with 30 purposively chosen informants were used. The results indicated that 70% of students perceive a paradigm shift in their learning experience before and after COVID-19. It was discovered that their sociodemographic characteristics influenced 60% of students, and 57% were influenced by their parent's socioeconomic status, which influenced their academic achievements during the COVID-19 outbreak. The findings revealed that 63% of students had faced numerous …

Designing Hybrid Human-Ai Orchestration Tools For Individual And Collaborative Activities: A Technology Probe Study, Vanessa Echeverria, Kexin Yang, Luettamae Lawrence, Nikol Rummel, Vincent Aleven

Instructional Technology and Learning Sciences Faculty Publications

Combining individual and collaborative learning is common, but dynamic combinations (which happen as-the-need arises, rather than in pre-planned ways, and may happen on an individual basis) are rare. This work reports findings from a technology probe study exploring alternative designs for classroom co-orchestration support for dynamically transitioning between individual and collaborative learning. The study involved 1) a technology-probe classroom study in an authentic, AI-supported classroom to understand teachers' and students' needs for co-orchestration support over dynamic transitions; and 2) workshops and interviews with students and teachers to get informed feedback about their lived experiences. 118 students and three teachers from …

A Dashboard To Support Teachers During Students’ Self-Paced Ai-Supported Problem-Solving Practice, Vincent Aleven, Jori Blankestijn, Luettamae Lawrence, Tomohiro Nagashima, Niels Taatgen

Instructional Technology and Learning Sciences Faculty Publications

Past research has yielded ample knowledge regarding the design of analytics-based tools for teachers and has found beneficial effects of several tools on teaching and learning. Yet there is relatively little knowledge regarding the design of tools that support teachers when a class of students uses AI-based tutoring software for self-paced learning. To address this challenge, we conducted design-based research with 20 middle school teachers to create a novel real-time dashboard, Tutti, that helps a teacher monitor a class and decide which individual students to help, based on analytics from students’ tutoring software. Tutti is fully implemented and has been …

Positionality: The Interplay Of Space, Context And Identity, Rebecca Y. Bayeck

Instructional Technology and Learning Sciences Faculty Publications

This article considers the way in which positionality shifts and is formed during a cross-cultural study to reveal the complexity of the insider-outsider status. As a researcher in a male-dominated game setting, I reflect on the research process and my interactions with participants to show the interplay of space, context, and identity in shaping a researcher’s status. I discuss the process of gaining access to the research site and participants, and data collection in relation to space, context, and identity. The interaction of my identities with space, and context informed my status at various moments. This interplay constructs a complex …

Co-Designing Ai-Based Orchestration Tools To Support Dynamic Transitions: Design Narratives Through Conjecture Mapping, Luettamae Lawrence, Boyuan Guo, Kexin Yang, Vanessa Echeverria, Zimmy Kang, Vikrant Bathala, Christina Li

Instructional Technology and Learning Sciences Faculty Publications

Dynamically transitioning between individual and collaborative learning has been hypothesized to have positive effects, such as providing the optimal learning mode based on students’ needs. There are, however, challenges in orchestrating these transitions in real-time while managing a classroom of students. AI-based orchestration tools have the potential to alleviate some of the orchestration load for teachers. In this study, we describe a sequence of three design sessions with teachers where we refine prototypes of an orchestration tool to support dynamic transitions. We leverage design narratives and conjecture mapping for the design of our novel orchestration tool. Our contributions include the …

Mapping The Complexities And Benefits Of Research-Design Partnerships, Emma Mercier, Luettamae Lawrence, June Ahn, Christopher Wegemer, Maya Benichou, Yael Kali, Yotam Hod, Marcela Borge, Kimberley Gomez, Ung-Sang Lee, Susan Mckenney, Cindy Poortman, Paula Arce-Trigatti, Britte Haugan Cheng

Instructional Technology and Learning Sciences Faculty Publications

Increasingly our field is recognizing the necessity of close, collaborative relationships with educators, policy makers, students and other potential stakeholders if our design and research work is to have a lasting and more equitable impact on education. However, this work is not easy or quick, and we lack both detailed examples of how it is done and training for new (and current) scholars in how to do it. This symposium brings together a group of scholars who actively engage in RPPs and DBIR in order to highlight the lessons that have been learned and extend our discourse into the realities …

Ai And Ambitious Learning Practices, Cindy E. Hmelo-Silver, Sadhana Puntambekar, Krista Glazewski, Luettamae Lawrence, Nikol Rummel, Vincent Aleven, Gautam Biswas, Suraj Uttamchandani, Asmalina Saleh, Haesol Bae, Thomas Brush, Bradford Mott, James Lester, William Goss, Dana Gnesdilow, Rebecca Passonneau, Purushartha Singh, Chanmin Kim, Marcelo Worsley

Instructional Technology and Learning Sciences Faculty Publications

This symposium will provide opportunities for discussion about how Artificial Intelligence can support ambitious learning practices in CSCL. To the extent that CSCL can be a lever for educational equitable educational change, AI needs to be able to support the kinds of practices that afford agency to students and teachers. However, AI also brings to the fore the need to consider equity and ethics. This interactive session will provide opportunities to discuss these issues in the context of the examples presented here.

Surveying Teachers' Preferences And Boundaries Regarding Human-Ai Control In Dynamic Pairing Of Students For Collaborative Learning, Kexin Bella Yang, Luettamae Lawrence, Vanessa Echeverría, Boyuan Guo, Nikol Rummel, Vincent Aleven

Instructional Technology and Learning Sciences Faculty Publications

Orchestration tools may support K-12 teachers in facilitating student learning, especially when designed to address classroom stakeholders’ needs. Our previous work revealed a need for human-AI shared control when dynamically pairing students for collaborative learning in the classroom, but offered limited guidance on the role each agent should take. In this study, we designed storyboards for scenarios where teachers, students and AI co-orchestrate dynamic pairing when using AI-based adaptive math software for individual and collaborative learning. We surveyed 54 math teachers on their co-orchestration preferences. We found that teachers would like to share control with the AI to lessen their …

In-Person Prehealth Advising: Impact Analysis Spring 2017 To Fall 2020, Alasdair Ekpenyong, Mykel Beorchia

Publications

At Utah State University, some of the advising department’s efforts specifically focus on preparing students for study in health professions graduate school. Students considering a career in the health sciences meet with an advisor who has been trained on the nuances of preparing for health professions graduate school.

This report explored the association between in-person prehealth advising participation and student persistence to the next term.

The Intersection Of Cultural Context And Research Encounter: Focus On Interviewing In Qualitative Research, Rebecca Yvonne Bayeck

Instructional Technology and Learning Sciences Faculty Publications

This article discusses the influence of the cultural context on the interview process. With literature demonstrating the role of spatial context on interviews, the article contends that similar consideration should be given to cultural contexts of research studies. Focusing on the cultural context where the interview takes place and the interactions during the interview can help researchers understand and analyze interview material. Interview forms such as conversation/interview bombing emerged from the interaction of cultural context with the interview process. This points to the need for qualitative researchers to explore how the cultural context shapes their research encounter. Such focus will …

Online Prehealth Advising: Impact Analysis Spring 2017 To Fall 2020, Alasdair Ekpenyong, Mykel Beorchia

Publications

At Utah State University, various online, Canvas-based advising programs complement the traditional in-person advising program. The online prehealth advising service assists students who are considering health professions graduate school.

This report explored the association between online prehealth advising participation and student persistence to the next term at Utah State University.

Examining The Influence Of Instructor Interventions On Group Collaboration, Luettamae Lawrence, Taylor Tucker, Emma Mercier

Instructional Technology and Learning Sciences Faculty Publications

Collaborative problem solving is often used in STEM higher education courses to support conceptual knowledge and teamwork. However, course teaching assistants (TAs) often lack the collaborative pedagogical knowledge necessary to orchestrate this form of learning. In this paper, we examine TAs’ orchestration strategies and technology used to understand how these factors influence groups’ collaboration. Contributions from this paper describe the interplay among technology, strategies, and groups’ collaboration toward understanding how to support collaboration in these courses.

Towards Bringing Human-Centered Design To K-12 And Post-Secondary Education, Saadeddine Shehab, Mike Tissenbaum, Luettamae Lawrence, Daniel Rees Lewis, Matthew Easterday, Spencer Carlson, Adam Royalty, Helen Chen, Sheppard Sheri, Shelley Goldman, Annie Camey Kuo, Kimiko Lange, Melissa Mesinas, Rose K. Pozos, Dhvani Toprani, Mona Alqahtani, Yu Xia, Marcela Borge, Keith Sawyer

Instructional Technology and Learning Sciences Faculty Publications

Human-Centered Design (HCD) is a growing field that has the potential to positively impact students’ learning. A general consensus on the terms, practices, scaffolds, and assessments of HCD can foster its effective implementation in K-12 and post-secondary education. This session brings together researchers whose work is focused on implementing HCD across K-16 classrooms. It aims to develop a coherent definition of HCD, its methods, practices, and assessments, to help frame the field and reduce ambiguity at a critical time in its broader adoption.

Explorations Of Designing Spatial Classroom Analytics With Virutal Prototyping, Jiwoong Jang, Jaewook Lee, Vanessa Echeverría, Luettamae Lawrence, Vincent Aleven

Instructional Technology and Learning Sciences Faculty Publications

Despite the potential of spatial displays for supporting teachers’ classroom orchestration through real-time classroom analytics, the process to design these displays is a challenging and under-explored topic in the learning analytics (LA) community. This paper proposes a mid-fidelity Virtual Prototyping method (VPM), which involves simulating a classroom environment and candidate designs in virtual space to address these challenges. VPM allows for rapid prototyping of spatial features, requires no specialized hardware, and enables teams to conduct remote evaluation sessions. We report observations and findings from an initial exploration with five potential users through a design process utilizing VPM to validate designs …

Understanding The Transient Nature Of Stem Doctoral Students’ Research Self-Efficacy Across Time: Considering The Role Of Gender, Race, And First-Generation College Status, Kaylee Litson, Jennifer M. Blaney, David F. Feldon

Instructional Technology and Learning Sciences Faculty Publications

Developing research self-efficacy is an important part of doctoral student preparation. Despite the documented importance of research self-efficacy, little is known about the progression of doctoral students’ research self-efficacy over time in general and for students from minoritized groups. This study examined both within- and between-person stability of research self-efficacy from semester to semester over 4 years, focusing on doctoral students in biological sciences (N = 336). Using random intercept autoregressive analyses, we evaluated differences in stability across gender, racially minoritized student status, and first-generation student status. Results showed similar mean levels of self-efficacy across demographic groups and across …

Communicating About Computational Thinking: Understanding Affordances Of Portfolios For Assessing High School Students’ Computational Thinking And Participation Practices, Deborah A. Fields, Debora Lui, Yasmin Kafai, Gayithri Jayathirtha, Justice Walker, Mia Shaw

Instructional Technology and Learning Sciences Faculty Publications

Background and Context: While assessment of computational thinking concepts, practices, and perspectives is at the forefront of K-12 CS education, supporting student communication about computation has received relatively little attention.

Objective: To examine the usability of process-based portfolios for capturing students’ communication about their computational practices regarding the process of making electronic textile projects.

Method: We examined the portfolios of 248 high school students in 15 introductory CS classrooms from largely underserved communities, using a formal rubric (top-down) to code computational communication and an open-coding scheme (bottom-up) to identify computational practices described.

Findings: Students demonstrated stronger …

Debugging By Design: A Constructionist Approach To High School Students’ Crafting And Coding Of Electronic Textiles As Failure Artifacts, Deborah A. Fields, Yasmin B. Kafai, Luis Morales-Navarro, Justice T. Walker

Instructional Technology and Learning Sciences Faculty Publications

Much attention in constructionism has focused on designing tools and activities that support learners in designing fully finished and functional applications and artifacts to be shared with others. But helping students learn to debug their applications often takes on a surprisingly more instructionist stance by giving them checklists, teaching them strategies or providing them with test programs. The idea of designing bugs for learning—or debugging by design—makes learners agents of their own learning and, more importantly, of making and solving mistakes. In this paper, we report on our implementation of “Debugging by Design” activities in a high school classroom over …

Connections Impact On Student Persistence: Impact Report Spring 2015 To Fall 2018, Amanda M. Hagman, Heidi Kesler, Matt Sanders, Mitchell Colver

Publications

Connections is Utah State University's (USU) first-year seminary. A primary objective of Connections is student persistence. It is designed to help students become learners. While being a learner is not synonymous with being a college student, it aligns students’ expectations with what is required to succeed in college and at USU. This impact report explores the influence of Connections participation on student persistence to the next term. Participation in Connections is associated with a 1.4% increase in persistence to the next term. The positive impact of Connections is increasing with strategic programmatic changes.

Passport Experience: Impact Analytics Fall 2014 To Fall 2018, Amanda M. Hagman, Nate Jensen, Lisa Simmons, Janet Anderson, Erik Dickamore

Publications

Utah State University (USU) dedicates substantial resources to support student transition to higher education. The Passport Experience cuts across all university domains to support early student participation in curricular, co-curricular, and extra-curricular activities. Students are invited to attend a variety of events, when milestones are reached, students are rewarded. Persistence is a primary objective of the Passport Experience. The Passport Experience helps students develop an increased awareness of campus events, broad their engagement in the university experience, and become more involved in the University community. This report explores the association between the Passport Experience and students’ persistence toward graduation. METHODS: …

How Do Usu Students Use University Support Services?, Amanda M. Hagman, Meghan Lewis, Kristi Swainston, Mitchell Colver

Publications

The central mission of USU is to be one of the nation’s premier student-centered universities. This is accomplished by fostering academics and offering opportunities to expand students’ vision of the world around them through co- and extra-curricular activities. These auxiliary activities are designed to support academic achievement and to engage students in meaningful opportunities to practice and enhance their personal and professional well-being. This analysis describes how students are using co- and extra-curricular services. It investigates the most salient student-facing programming in supporting student retention.

Writing Fellows: Impact Analysis Fall 2015 To Spring 2019, Amanda M. Hagman, Julie Foust, Erik Dickamore, Hayden Hoopes

Publications

The Writing Fellows program strategically places high performing writing mentors in courses with rigorous writing requirements. Writing Fellows work with each student in a course by reviewing their writing and offering mentoring to improve their written communication skills. Persistence is a secondary objective of the Writing Fellow program. As such, an impact evaluation on persistence should only be used as part of an evaluation of the influence of the Writing Fellows program on student wellbeing. This impact evaluation on student persistence found that students in courses with a Writing Fellow experienced a significant increase in persistence to the next term …

Writing Center: Impact Analysis Fall 2014 To Fall 2018, Amanda M. Hagman, Kendall Becker, Susan B. Andersen, Star Coulbrooke, Erik Dickamore, Jasilyn Heaps

Publications

The Utah State University (USU) Writing Center is dedicated to empowering students to express their knowledge and ideas in writing. Their approach promotes academic inquiry, critical thinking, and expressions of diversity. While research and evaluation suggest that the Writing Center significantly impacts student academic performance, the impact on student persistence is not yet clear. This report explores the association between USU’s Writing Center and students’ persistence toward graduation. METHODS: Students’ Writing Center use was captured through student log-ins at writing appointments. Students who had a record of using the Writing Center were compared to similar students who did not have …

New Student Orientation: Impact Analysis Fall 2017 To Spring 2019, Amanda M. Hagman, Jacob Van Den Akker, Erik Dickamore, Lisa Simmons

Publications

The transition to university represents a major life change more incoming students. To facilitate this transition, USU adopted a phased-orientation system that provides students information justin- time for use. Currently, Module 3 can be completed either in-person or online. This analysis explores both the impact of the phased-orientation modules and the impact of completing Module 3 online verse in-person on student persistence to the next term. METHODS: First students who completed Module 3 online were compared to students who completed Module 3 in-person. Next students who completed all 5 modules were compared to students who only completed the required modules. …