Education Commons^™

How Teachers Conceptualise Shared Control With An Ai Co-Orchestration Tool: A Multiyear Teacher-Centred Design Process, Luettamae Lawrence, Vanessa Echeverria, Kexin Yang, Vincent Aleven, Nikol Rummel

Instructional Technology and Learning Sciences Faculty Publications

Artificial intelligence (AI) can enhance teachers' capabilities by sharing control over different parts of learning activities. This is especially true for complex learning activities, such as dynamic learning transitions where students move between individual and collaborative learning in un-planned ways, as the need arises. Yet, few initiatives have emerged considering how shared responsibility between teachers and AI can support learning and how teachers' voices might be included to inform design decisions. The goal of our article is twofold. First, we describe a secondary analysis of our co-design process comprising six design methods to understand how teachers conceptualise sharing control with …

How Co-Designing Computational Modeling Activities Helped Teachers Implement Responsive Teaching Strategies, Hillary Swanson, Luettamae Lawrence, Jared Arnell, Allisia Dawkins, Bonni Jones, Bruce Sherin, Uri Wilensky

Instructional Technology and Learning Sciences Faculty Publications

In recent years, science education has shifted focus, from content to practice. This is reflected in the NGSS, which advocate learning science concepts through engagement in science and engineering practices. Theory building is a central activity of science and computational modeling is a key practice through which contemporary scientists construct theory. In this paper, we discuss an 8th grade science teacher's implementation of a computational modeling lesson. The teacher had co-designed the computational modeling microworld and lesson with the research team over the preceding summers. We investigate the teacher's activity during a whole-class discussion near the end of the lesson, …

Understanding Computational Thinking In The Gameplay Of The African Songo Board Game, Rebecca Y. Bayeck

Instructional Technology and Learning Sciences Faculty Publications

Computational thinking is a necessary skill for the 21st century. While previously examined in computer-rich settings, researchers are increasingly studying computational thinking in unplugged environments such as board games. Focusing on the African board game Songo, this study shows that computational thinking practices are embedded in Songo board gameplay and interact with the cultural context. The study also reveals a computing practice peculiar to Songo gameplay, songoputation. This paper suggests that researchers can benefit from exploring computational thinking and computing practices beyond board games in western contexts.

Supporting And Sustaining Equitable Steam Activities In High School Classrooms: Understanding Computer Science Teachers’ Needs And Practices When Implementing An E-Textiles Curriculum To Forge Connections Across Communities, Deborah A. Fields, Yasmin B. Kafai

Instructional Technology and Learning Sciences Faculty Publications

While the last two decades have seen an increased interest in STEAM (science, technology, engineering, arts, and mathematics) in K-12 schools, few efforts have focused on the teachers and teaching practices necessary to support these interventions. Even fewer have considered the important work that teachers carry out not just inside classrooms but beyond the classroom walls to sustain such STEAM implementation efforts, from interacting with administrators to recruiting students and persuading parents about the importance of arts and computer science. In order to understand teachers’ needs and practices regarding STEAM implementation, in this paper, we focus on eight experienced computer …

Is Microethnography An Ethnographic Case Study? And/Or A Mini-Ethnographic Case Study? An Analysis Of The Literature, Rebecca Y. Bayeck

Instructional Technology and Learning Sciences Faculty Publications

Selecting the research approach that addresses the research question is often challenging for novice researchers. However, getting a better understanding of the research approaches available in the field, is likely to help novice researchers identify and choose the research approach that fits their situation. In this paper, we discuss microethnography, ethnographic case study, and mini-ethnography case study in order to show that these approaches may have similarities but are different. The author hopes that this discussion will help researchers get a better understanding of these approaches and dissipate the confusion that may exist.

The Predictive Validity Of The Gre Across Graduate Outcomes: A Meta-Analysis Of Trends Over Time, David F. Feldon, Kaylee Litson, Brinleigh Cahoon, Zhang Feng, Andrew Walker, Colby Tofel-Grehl

Instructional Technology and Learning Sciences Faculty Publications

This meta-analysis assesses the predictive validity of the Graduate Record Examination (GRE) across outcome variables, including grade point average, for graduate students. In addition to aggregate effects, this paper also assessed changes in observed effects over time as related to increasing diversity in the graduate student population and as a function of gender and racial/ethnic composition of study samples. Framed using a lens of critical whiteness, this analysis examined n = 1,744 individual effects across k = 208 studies. Overall, 61.6% of reported effects were nonsignificant (i.e. no predictive value of GRE scores on student outcomes). Further, the magnitude of …

Designing Hybrid Human-Ai Orchestration Tools For Individual And Collaborative Activities: A Technology Probe Study, Vanessa Echeverria, Kexin Yang, Luettamae Lawrence, Nikol Rummel, Vincent Aleven

Instructional Technology and Learning Sciences Faculty Publications

Combining individual and collaborative learning is common, but dynamic combinations (which happen as-the-need arises, rather than in pre-planned ways, and may happen on an individual basis) are rare. This work reports findings from a technology probe study exploring alternative designs for classroom co-orchestration support for dynamically transitioning between individual and collaborative learning. The study involved 1) a technology-probe classroom study in an authentic, AI-supported classroom to understand teachers' and students' needs for co-orchestration support over dynamic transitions; and 2) workshops and interviews with students and teachers to get informed feedback about their lived experiences. 118 students and three teachers from …

A Dashboard To Support Teachers During Students’ Self-Paced Ai-Supported Problem-Solving Practice, Vincent Aleven, Jori Blankestijn, Luettamae Lawrence, Tomohiro Nagashima, Niels Taatgen

Instructional Technology and Learning Sciences Faculty Publications

Past research has yielded ample knowledge regarding the design of analytics-based tools for teachers and has found beneficial effects of several tools on teaching and learning. Yet there is relatively little knowledge regarding the design of tools that support teachers when a class of students uses AI-based tutoring software for self-paced learning. To address this challenge, we conducted design-based research with 20 middle school teachers to create a novel real-time dashboard, Tutti, that helps a teacher monitor a class and decide which individual students to help, based on analytics from students’ tutoring software. Tutti is fully implemented and has been …

Positionality: The Interplay Of Space, Context And Identity, Rebecca Y. Bayeck

Instructional Technology and Learning Sciences Faculty Publications

This article considers the way in which positionality shifts and is formed during a cross-cultural study to reveal the complexity of the insider-outsider status. As a researcher in a male-dominated game setting, I reflect on the research process and my interactions with participants to show the interplay of space, context, and identity in shaping a researcher’s status. I discuss the process of gaining access to the research site and participants, and data collection in relation to space, context, and identity. The interaction of my identities with space, and context informed my status at various moments. This interplay constructs a complex …

Co-Designing Ai-Based Orchestration Tools To Support Dynamic Transitions: Design Narratives Through Conjecture Mapping, Luettamae Lawrence, Boyuan Guo, Kexin Yang, Vanessa Echeverria, Zimmy Kang, Vikrant Bathala, Christina Li

Instructional Technology and Learning Sciences Faculty Publications

Dynamically transitioning between individual and collaborative learning has been hypothesized to have positive effects, such as providing the optimal learning mode based on students’ needs. There are, however, challenges in orchestrating these transitions in real-time while managing a classroom of students. AI-based orchestration tools have the potential to alleviate some of the orchestration load for teachers. In this study, we describe a sequence of three design sessions with teachers where we refine prototypes of an orchestration tool to support dynamic transitions. We leverage design narratives and conjecture mapping for the design of our novel orchestration tool. Our contributions include the …

Mapping The Complexities And Benefits Of Research-Design Partnerships, Emma Mercier, Luettamae Lawrence, June Ahn, Christopher Wegemer, Maya Benichou, Yael Kali, Yotam Hod, Marcela Borge, Kimberley Gomez, Ung-Sang Lee, Susan Mckenney, Cindy Poortman, Paula Arce-Trigatti, Britte Haugan Cheng

Instructional Technology and Learning Sciences Faculty Publications

Increasingly our field is recognizing the necessity of close, collaborative relationships with educators, policy makers, students and other potential stakeholders if our design and research work is to have a lasting and more equitable impact on education. However, this work is not easy or quick, and we lack both detailed examples of how it is done and training for new (and current) scholars in how to do it. This symposium brings together a group of scholars who actively engage in RPPs and DBIR in order to highlight the lessons that have been learned and extend our discourse into the realities …

Ai And Ambitious Learning Practices, Cindy E. Hmelo-Silver, Sadhana Puntambekar, Krista Glazewski, Luettamae Lawrence, Nikol Rummel, Vincent Aleven, Gautam Biswas, Suraj Uttamchandani, Asmalina Saleh, Haesol Bae, Thomas Brush, Bradford Mott, James Lester, William Goss, Dana Gnesdilow, Rebecca Passonneau, Purushartha Singh, Chanmin Kim, Marcelo Worsley

Instructional Technology and Learning Sciences Faculty Publications

This symposium will provide opportunities for discussion about how Artificial Intelligence can support ambitious learning practices in CSCL. To the extent that CSCL can be a lever for educational equitable educational change, AI needs to be able to support the kinds of practices that afford agency to students and teachers. However, AI also brings to the fore the need to consider equity and ethics. This interactive session will provide opportunities to discuss these issues in the context of the examples presented here.

Surveying Teachers' Preferences And Boundaries Regarding Human-Ai Control In Dynamic Pairing Of Students For Collaborative Learning, Kexin Bella Yang, Luettamae Lawrence, Vanessa Echeverría, Boyuan Guo, Nikol Rummel, Vincent Aleven

Instructional Technology and Learning Sciences Faculty Publications

Orchestration tools may support K-12 teachers in facilitating student learning, especially when designed to address classroom stakeholders’ needs. Our previous work revealed a need for human-AI shared control when dynamically pairing students for collaborative learning in the classroom, but offered limited guidance on the role each agent should take. In this study, we designed storyboards for scenarios where teachers, students and AI co-orchestrate dynamic pairing when using AI-based adaptive math software for individual and collaborative learning. We surveyed 54 math teachers on their co-orchestration preferences. We found that teachers would like to share control with the AI to lessen their …

The Intersection Of Cultural Context And Research Encounter: Focus On Interviewing In Qualitative Research, Rebecca Yvonne Bayeck

Instructional Technology and Learning Sciences Faculty Publications

This article discusses the influence of the cultural context on the interview process. With literature demonstrating the role of spatial context on interviews, the article contends that similar consideration should be given to cultural contexts of research studies. Focusing on the cultural context where the interview takes place and the interactions during the interview can help researchers understand and analyze interview material. Interview forms such as conversation/interview bombing emerged from the interaction of cultural context with the interview process. This points to the need for qualitative researchers to explore how the cultural context shapes their research encounter. Such focus will …

Examining The Influence Of Instructor Interventions On Group Collaboration, Luettamae Lawrence, Taylor Tucker, Emma Mercier

Instructional Technology and Learning Sciences Faculty Publications

Collaborative problem solving is often used in STEM higher education courses to support conceptual knowledge and teamwork. However, course teaching assistants (TAs) often lack the collaborative pedagogical knowledge necessary to orchestrate this form of learning. In this paper, we examine TAs’ orchestration strategies and technology used to understand how these factors influence groups’ collaboration. Contributions from this paper describe the interplay among technology, strategies, and groups’ collaboration toward understanding how to support collaboration in these courses.

Towards Bringing Human-Centered Design To K-12 And Post-Secondary Education, Saadeddine Shehab, Mike Tissenbaum, Luettamae Lawrence, Daniel Rees Lewis, Matthew Easterday, Spencer Carlson, Adam Royalty, Helen Chen, Sheppard Sheri, Shelley Goldman, Annie Camey Kuo, Kimiko Lange, Melissa Mesinas, Rose K. Pozos, Dhvani Toprani, Mona Alqahtani, Yu Xia, Marcela Borge, Keith Sawyer

Instructional Technology and Learning Sciences Faculty Publications

Human-Centered Design (HCD) is a growing field that has the potential to positively impact students’ learning. A general consensus on the terms, practices, scaffolds, and assessments of HCD can foster its effective implementation in K-12 and post-secondary education. This session brings together researchers whose work is focused on implementing HCD across K-16 classrooms. It aims to develop a coherent definition of HCD, its methods, practices, and assessments, to help frame the field and reduce ambiguity at a critical time in its broader adoption.

Explorations Of Designing Spatial Classroom Analytics With Virutal Prototyping, Jiwoong Jang, Jaewook Lee, Vanessa Echeverría, Luettamae Lawrence, Vincent Aleven

Instructional Technology and Learning Sciences Faculty Publications

Despite the potential of spatial displays for supporting teachers’ classroom orchestration through real-time classroom analytics, the process to design these displays is a challenging and under-explored topic in the learning analytics (LA) community. This paper proposes a mid-fidelity Virtual Prototyping method (VPM), which involves simulating a classroom environment and candidate designs in virtual space to address these challenges. VPM allows for rapid prototyping of spatial features, requires no specialized hardware, and enables teams to conduct remote evaluation sessions. We report observations and findings from an initial exploration with five potential users through a design process utilizing VPM to validate designs …

Understanding The Transient Nature Of Stem Doctoral Students’ Research Self-Efficacy Across Time: Considering The Role Of Gender, Race, And First-Generation College Status, Kaylee Litson, Jennifer M. Blaney, David F. Feldon

Instructional Technology and Learning Sciences Faculty Publications

Developing research self-efficacy is an important part of doctoral student preparation. Despite the documented importance of research self-efficacy, little is known about the progression of doctoral students’ research self-efficacy over time in general and for students from minoritized groups. This study examined both within- and between-person stability of research self-efficacy from semester to semester over 4 years, focusing on doctoral students in biological sciences (N = 336). Using random intercept autoregressive analyses, we evaluated differences in stability across gender, racially minoritized student status, and first-generation student status. Results showed similar mean levels of self-efficacy across demographic groups and across …

Communicating About Computational Thinking: Understanding Affordances Of Portfolios For Assessing High School Students’ Computational Thinking And Participation Practices, Deborah A. Fields, Debora Lui, Yasmin Kafai, Gayithri Jayathirtha, Justice Walker, Mia Shaw

Instructional Technology and Learning Sciences Faculty Publications

Background and Context: While assessment of computational thinking concepts, practices, and perspectives is at the forefront of K-12 CS education, supporting student communication about computation has received relatively little attention.

Objective: To examine the usability of process-based portfolios for capturing students’ communication about their computational practices regarding the process of making electronic textile projects.

Method: We examined the portfolios of 248 high school students in 15 introductory CS classrooms from largely underserved communities, using a formal rubric (top-down) to code computational communication and an open-coding scheme (bottom-up) to identify computational practices described.

Findings: Students demonstrated stronger …

Debugging By Design: A Constructionist Approach To High School Students’ Crafting And Coding Of Electronic Textiles As Failure Artifacts, Deborah A. Fields, Yasmin B. Kafai, Luis Morales-Navarro, Justice T. Walker

Instructional Technology and Learning Sciences Faculty Publications

Much attention in constructionism has focused on designing tools and activities that support learners in designing fully finished and functional applications and artifacts to be shared with others. But helping students learn to debug their applications often takes on a surprisingly more instructionist stance by giving them checklists, teaching them strategies or providing them with test programs. The idea of designing bugs for learning—or debugging by design—makes learners agents of their own learning and, more importantly, of making and solving mistakes. In this paper, we report on our implementation of “Debugging by Design” activities in a high school classroom over …

Leveraging Local Resources And Contexts For Inclusive Computer Science Classrooms: Reflections From Experienced High School Teachers Implementing Electronic Textiles, Mia S. Shaw, Deborah A. Fields, Yasmin B. Kafai

Instructional Technology and Learning Sciences Faculty Publications

Background and context

Promoting open-ended projects presents new opportunities and challenges for inclusive teaching in CS classrooms. While efforts have been made to develop inclusive curricula, little research has focused on ways teachers apply curricula in their classrooms to promote inclusion.

Objective

To understand the challenges faced in facilitating an open-ended unit and the pedagogical strategies enacted to address those challenges, we analyze the self-reported teaching practices that experienced teachers developed in their implementation of a constructionist electronic textiles unit in Exploring Computer Science.

Method

We inductively analyzed and coded 17 experienced teachers’ weekly surveys and post-interviews.

Findings

Teachers …

Finding A Fit: Biological Science Doctoral Students’ Selection Of A Principal Investigator And Research Laboratory, Michelle A. Maher, Annie M. Wofford, Josipa Roksa, David F. Feldon

Instructional Technology and Learning Sciences Faculty Publications

In the laboratory-based disciplines, selection of a principal investigator (PI) and research laboratory (lab) indelibly shapes doctoral students’ experiences and educational outcomes. Framed by the theoretical concept of person–environment fit from within a socialization model, we use an inductive, qualitative approach to explore how a sample of 42 early-stage doctoral students enrolled in biological sciences programs made decisions about fitting with a PI and within a lab. Results illuminated a complex array of factors that students considered in selecting a PI, including PI relationship, mentoring style, and professional stability. Further, with regard to students’ lab selection, peers and research projects …

Towards The Effective Implementation Of Collaborative Problem Solving In Undergraduate Engineering Classrooms: Co-Designing Guidelines For Teaching Assistants, Saadeddine Shehab, Luettamae Lawrence, Emma Mercier, Anthony Salvatore Margotta, Elizabeth Renee Livingston, Mariana Silva, Taylor Tucker

Instructional Technology and Learning Sciences Faculty Publications

Typically, in higher education STEM classes, teaching assistants (TAs) perform teaching duties such as leading and running discussion sections where students apply concepts they have encountered in lectures. Nevertheless, research studies show that TAs struggle to effectively implement collaborative problem solving activities. One contributing factor is the lack of actual guidelines that can help TAs translate the theoretical ideas about implementing collaborative problem solving into concrete actions. This paper presents guidelines that can be used by TAs to implement collaborative problem solving activities in undergraduate engineering classrooms. Three researchers and two engineering teaching assistants participated in two, two-hour long workshops …

“How Well Does Your Structural Equation Model Fit Your Data?”: Is Marcoulides And Yuan’S Equivalence Test The Answer?, James Peugh, David F. Feldon

Instructional Technology and Learning Sciences Faculty Publications

Structural equation modeling is an ideal data analytical tool for testing complex relationships among many analytical variables. It can simultaneously test multiple mediating and moderating relationships, estimate latent variables on the basis of related measures, and address practical issues such as nonnormality and missing data. To test the extent to which a hypothesized model provides an appropriate characterization of the collective relationships among its variables, researchers must assess the “fit” between the model and the sample’s data. However, interpreting estimates of model fit is a problematic process. The traditional inferential test of model fit, the chi-square test, is biased due …

Steam Maker Education: Conceal/Reveal Of Personal, Artistic And Computational Dimensions In High School Student Projects, Lindsay Lindberg, Deborah A. Fields, Yasmin B. Kafai

Instructional Technology and Learning Sciences Faculty Publications

Much of maker education’s expansion has focused on STEM (science, technology, engineering, mathematics) learning, leaving out equally promising opportunities for integrating arts learning. In this paper we share findings from a project in which high school students created electronic-textiles-based art representing features of a community important to them as a part of an elective high school computer science class. We addressed the following research questions: (1) What kinds of personal and community meanings did students convey through their maker projects? (2) How did students engage with artistic dimensions in their projects? (3) How did coding interactive features support students’ artwork? …

The Validity And Instructional Value Of A Rubric For Evaluating Online Course Quality: An Empirical Study, Ji Eun Lee, Mimi Recker, Min Yuan

Instructional Technology and Learning Sciences Faculty Publications

This study investigated the validity and instructional value of a rubric developed to evaluate the quality of online courses offered at a midsized public university. This rubric was adapted from an online course quality rubric widely used in higher education, the Quality Matters rubric. We first examined the reliability and preliminary construct validity of the rubric using quality ratings for 202 online courses and eliminated twelve problematic items. We then examined the instructional value of the rubric by investigating causal relationships between 1) course quality scores, 2) online interactions between students, instructors, and content, and 3) student course performance (course …

Consensual Assessment In The New Domain Of E-Textiles: Comparing Insights From Expert, Quasi-Expert, And Novice Judges, Anthony Phonethibsavads, Maggie Dahn, Kylie Peppler, Deborah A. Fields, Yasmin B. Kafai

Instructional Technology and Learning Sciences Faculty Publications

Establishing what constitutes creativity in a domain is something for which we often look to experts—individuals versed in a domain’s history and able to identify timeworn ideas from fresh ones. Such valuations of creative merit are tied to a familiarity with past and present trends and, therefore, opinions of newcomers are often ignored. However, what about domains that build upon new, unexplored practices? This study examines the creativity ratings of judges with varying expertise in the emergent domain of electronic textiles (or e-textiles). E-textiles are fabrics that have programmable electronics such as sensors and actuators embedded in them toward a …

Identifying Faculty And Peer Interaction Patterns Of First-Year Biology Doctoral Students: A Latent Class Analysis, Soojeong Jeong, Jennifer M. Blaney, David F. Feldon

Instructional Technology and Learning Sciences Faculty Publications

Faculty and peer interactions play a key role in shaping graduate student socialization. Yet, within the literature on graduate student socialization, researchers have primarily focused on understanding the nature and impact of faculty alone, and much less is known about how peer interactions also contribute to graduate student outcomes. Using a national sample of first-year biology doctoral students, this study reveals distinct categories that classify patterns of faculty and peer interaction. Further, we document inequities such that certain groups (e.g., underrepresented minority students) report constrained types of interactions with faculty and peers. Finally, we connect faculty and peer interaction patterns …

Co-Design Of An Orchestration Tool: Supporting Engineering Teaching Assistants As They Facilitate Collaborative Learning., Luettamae Lawrence, Emma Mercier

Instructional Technology and Learning Sciences Faculty Publications

This paper describes a design-based implementation research (DBIR) project, focused on the co-design and implementation of an orchestration tool for teaching assistants (TAs) in required engineering classes. Building on our collaboration with the engineering department, we identified a need for a tool that provides insight into groups to help TAs intervene in realtime. This paper presents two phases of our iterative co-design process. The first phase includes the initial design of the tool from design workshops with TAs. The second phase focuses on a 16-week implementation of the orchestration tool and reports on interviews with TAs to understand how they …

Communicating Computational Concepts And Practices Within High School Students’ Portfolios Of Making Electronic Textiles, Debora Lui, Justice T. Walker, Sheri Hanna, Yasmin B. Kafai, Deborah A. Fields, Gayithri Jayathirtha

Instructional Technology and Learning Sciences Faculty Publications

Portfolios have recently gained traction within computer science education as a way to assess students’ computational thinking and practices. Whereas traditional assessments such as exams tend to capture learning within artificial settings at a single point in time, portfolios provide more authentic opportunities to document a trajectory of students’ learning and practices in everyday contexts. Furthermore, because communication itself has been defined as an important computational thinking practice, portfolios give students a place to practice this skill in the classroom. In this study, we report on the implementation of a digital portfolio with a class of 21 high school students …