Education Commons^™

Toward Cs1 Content Subscales: A Mixed-Methods Analysis Of An Introductory Computing Assessment, Miranda C. Parker, Matt J. Davidson, Yvonne S. Kao, Lauren Margulieux, Zachary Tidler, Jan Vahrenhold

Learning Sciences Faculty Publications

Background and Context. There is a constant, demonstrated need for valid and reliable assessments in computing education research. While there exist assessments at a course-based level (e.g., CS1, Data Structures, Discrete math, etc.), instructors and researchers would also like concept-based subscales that are more fine-grained. However, assessments designed and validated at the course level need additional work to determine whether they can reliably and validly measure individual concepts.

Objectives. In this paper, we explore the content and factor structure of an existing CS1 assessment, the Second CS1 (SCS1) assessment, which consists of nine CS1 concepts and three question types (definitional, …

Interactive Transcription Techniques For Interaction Analysis, Arpit Mathur, Benjamin R. Shapiro

Learning Sciences Faculty Publications

Interaction analysis is a valuable method and approach to study knowledge in use in the learning sciences and CSCL communities. Central to interaction analysis is the creation of transcripts to selectively encode and represent audio and video data. However, current transcription techniques used in interaction analysis, including multimodal transcription techniques, have yet to explore the strengths and weaknesses of interactive visualization to selectively encode and represent people’s interaction in context. Drawing from our recent efforts to amplify, not automate, transcription in qualitative research, this paper interactively visualizes one video dataset in five different ways using contemporary interactive visualization techniques. Findings …

“Bettering Data”: The Role Of Everyday Language And Visualization In Critical Novice Data Work, Benjamin R. Shapiro, Amanda Meng, Annabel Rothschild, Sierra Gilliam, Cicely Garrett, Carl Disalvo, Betsy Disalvo

Learning Sciences Faculty Publications

Informed by critical data literacy efforts to promote social justice, this paper uses qualitative methods and data collected during two years of workplace ethnography to characterize the notion of critical novice data work. Specifically, we analyze everyday language used by novice data workers at DataWorks, an organization that trains and employs historically excluded populations to work with community data sets. We also characterize challenges faced by these workers in both cleaning and being critical of data during a project focused on police-community relations. Finally, we highlight novel approaches to visualizing data the workers developed during this project, derived from data …

When Wrong Is Right: The Instructional Power Of Multiple Conceptions, Lauren Margulieux, Paul Denny, Kathryn Cunningham, Michael Deutsch, Benjamin R. Shapiro

Learning Sciences Faculty Publications

For many decades, educational communities, including computing education, have debated the value of telling students what they need to know (i.e., direct instruction) compared to guiding them to construct knowledge themselves (i.e., constructivism). Comparisons of these two instructional approaches have inconsistent results. Direct instruction can be more efficient for short-term performance but worse for retention and transfer. Constructivism can produce better retention and transfer, but this outcome is unreliable. To contribute to this debate, we propose a new theory to better explain these research results. Our theory, multiple conceptions theory, states that learners develop better conceptual knowledge when they are …

Scaffolding Problem Solving With Learners’ Own Self Explanations Of Subgoals, Lauren Margulieux, Richard Catrambone

Learning Sciences Faculty Publications

Procedural problem solving is an important skill in most technical domains, like programming, but many students reach problem solving impasses and flounder. In most formal learning environments, instructors help students to overcome problem solving impasses by scaffolding initial problem solving. Relying on this type of personalized interaction, however, limits the scale of formal instruction in technical domains, or it limits the efficacy of learning environments without it, like many scalable online learning environments. The present experimental study explored whether learners’ self-explanations of worked examples could be used to provide personalized but non-adaptive scaffolding during initial problem solving to improve later …

Exploring Approaches To Data Literacy Through A Critical Race Theory Perspective, Britney Johnson, Benjamin R. Shapiro, Betsy Disalvo, Annabel Rothschild, Carl Disalvo

Learning Sciences Faculty Publications

In this paper, we describe and analyze a workshop developed for a work training program called DataWorks. In this
workshop, data workers chose a topic of their interest, sourced and processed data on that topic, and used that data to create
presentations. Drawing from discourses of data literacy; epistemic agency and lived experience; and critical race theory, we analyze the workshops’ activities and outcomes. Through this analysis, three themes emerge: the tensions between epistemic agency and the context of work, encountering the ordinariness of racism through data work, and understanding the personal as communal and intersectional. Finally, critical race theory …

Classroom Interaction Geography: Visualizing Space & Time In Classroom Interaction, Benjamin R. Shapiro, Brette Garner

Learning Sciences Faculty Publications

Methods to transcribe and represent classroom video data are central to studying teaching and learning in classrooms. However, current methods focus on encoding and representing data over time, not space. In this paper, we demonstrate the value of a new methodological approach called interaction geography to transcribe and interactively visualize classroom video data over space and time. We use interaction geography to illustrate classroom participation patterns in two case
studies from teacher education research that, until now, have been challenging to see. Findings characterize strengths, limitations, and next steps to expand interaction geography in classroom research and suggest new questions …

What About Interaction Geography To Evaluate Physical Learning Spaces?, Benjamin R. Shapiro

Learning Sciences Faculty Publications

This paper reviews and explores how interaction geography, a new approach to visualize people’s interaction over space and time, extends current approaches to evaluate physical learning spaces. This chapter begins by reviewing representations produced using interaction geography to study visitor engagement and learning in a museum. In particular, this review illustrates Mondrian Transcription, a method to map people’s movement and conversation over space and time, and the Interaction Geography Slicer (IGS), a dynamic visualisation tool that supports new forms of interaction and multi-modal analysis. Subsequently, this chapter explores how interaction geography may advance the evaluation of physical learning …

A Remote Instructor Like Me: Student-Teacher Congruence In Online, High School Courses, Jennifer Darling-Aduana

Learning Sciences Faculty Publications

Students belonging to marginalized groups experience positive impacts when taught by a teacher of the same race, ethnicity, and gender. The unique nature of standardized, asynchronous online course taking allows for greater separation of any possible educational benefits of student versus teacher-driven mechanisms contributing to these improved outcomes. Using a student-bycourse fixed effect strategy on data from a large urban school district, I examined associations between whether students experienced racial/ethnic or gender congruence with their remote instructor and both engagement and learning outcomes. Students who identified as Black demonstrated higher rates of engagement, although no difference in achievement, within lessons …

I Am Not Only A Student-Athlete: Investigating Social Identity Complexity As A Stereotype Threat Mitigation Strategy To Reduce Barriers, Jacob Alan English, Ann Cale Kruger

Learning Sciences Faculty Publications

Collegiate athletes must contend with harmful stereotypes (e.g., intellectually lazy, unintelligent) during their academic careers (Comeaux, 2012). Research shows that student-athletes’ academic performance can be negatively impacted by stereotype threat (Riciputi & Erdal, 2017). Currently, there is no published evidence-based research on stereotype threat (ST) mitigation strategies targeted to student-athletes. Expanding the work of Gresky and colleagues (2005), this study explored a self-concept map activity, based on the social identity complexity theory, as one potential strategy for collegiate athletes. Seventy Division I athletes were randomly assigned to one of three experimental conditions (varying by the level of ST administered). ANOVA …

The Curious Case Of Loops, Briana Baker Morrison, Lauren Margulieux, Adrienne Decker

Learning Sciences Faculty Publications

Background and Context: Subgoal labeled worked examples are effective for teaching computing concepts, but the research to date has been reported in a piecemeal fashion. This paper aggregates data from three studies, including data that has not been previously reported upon, to examine more holistically the effect of subgoal labeled worked examples across three student populations and across different instructional designs.

Objective: By aggregating the data, we provide more statistical and explanatory power for somewhat surprising yet replicable results. We discuss which results generalize across populations, focusing on a stable effect size to be expected when using subgoal labels in …

What Do We Think We Think We Are Doing?: Metacognition And Self-Regulation In Programming, James Prather, Brett A. Becker, Michelle Craig, Paul Denny, Dastyni Loksa, Lauren Margulieux

Learning Sciences Faculty Publications

Metacognition and self-regulation are popular areas of interest in programming education, and they have been extensively researched outside of computing. While computing education researchers should draw upon this prior work, programming education is unique enough that we should explore the extent to which prior work applies to our context. The goal of this systematic review is to support research on metacognition and self-regulation in programming education by synthesizing relevant theories, measurements, and prior work on these topics. By reviewing papers that mention metacognition or self-regulation in the context of programming, we aim to provide a benchmark of our current progress …

Effect Of Implementing Subgoals In Code.Org’S Intro To Programming Unit In Computer Science Principles, Lauren Margulieux, Briana Baker Morrison, Baker Franke, Harivololona Ramilison

Learning Sciences Faculty Publications

The subgoal learning framework has improved performance for novice programmers in higher education, but it has only started to be applied and studied in K-12 (primary/secondary). Programming education in K-12 is growing, and many international initiatives are attempting to increase participation, including curricular initiatives like Computer Science Principles and non-profit organizations like Code.org. Given that subgoal learning is designed to help students with no prior knowledge, we designed and implemented subgoals in the introduction to programming unit in Code.org’s Computer Science Principles course. The redesigned unit includes subgoal-oriented instruction and subgoal-themed pre-written comments that students could add to their programming …

Classroom Interaction Geography: A Case Study, Benjamin R. Shapiro, Brette Garner, Hui Soo Chae, Gary Natriello

Learning Sciences Faculty Publications

The study of classroom discourse is central to understanding and supporting effective teaching practice. Recently, researchers have begun to explore the spatial dimension of classroom discourse. However, this work emphasizes the lack of methods, particularly visual methods, to fully explore the spatial dimension of classroom discourse. This paper uses an approach to studying collaborative interaction we have developed called interaction geography to revisit a classic case known as “Sean Numbers” from the work of renown teacher educator Deborah Ball. Our analysis highlights the value of interaction geography to visually and dynamically explore the spatial and temporal dimensions of classroom discourse. …

Visualizing Qualitative Data: Creative Approaches For Analyzing And Demonstrating Lively Data From Diverse Learning Settings, Yong Ju Jung, Jaclyn Dudek, Shulong Yan, Marcela Borge, Soo Hyeon Kim, Jian Liao, Benjamin R. Shapiro, Heather Toomey Zimmerman

Learning Sciences Faculty Publications

This structured poster session aims to showcase novel approaches of qualitatively
analyzing and communicating lively data—data that is complex, nuanced, multimodal, and multi-voiced. Such data is rich but also messy, often defying the traditional text-based forms of description and presentation. Therefore, the session pairs creative techniques and methods to analyze, triangulate, and/or visualize qualitative findings across multiple data sources (e.g., video, digital and physical spaces, participant artifacts, and patterns of movement) from diverse learning contexts (e.g., museums, libraries, outdoor spaces, and classrooms)—beyond showing transcriptions. The visual format of the session supports our goal of sharing and communicating rich data stories …

Here And Then: Learning By Making Places With Digital Spatial Story Lines, Rogers Hall, Benjamin R. Shapiro, Andrew L. Hostetler, Helen Collins, David Owens, Fisher Douglas

Learning Sciences Faculty Publications

In this article, we introduce and analyze learning experiences made possible by a teaching framework that we have developed and call digital spatial story lines (DSSLs). DSSLs offer a novel approach to learning on the move by engaging learners with related conceptual practices of archival curation, digital mapping, and the production of public history. Learners collaborate to make and follow map-based story lines that bridge archival media they curate in public libraries and museums onto city neighborhoods these media describe. Story lines can be followed as tours to explore under- or untold stories about a city’s public history at walking …

Re-Shape: A Method To Teach Data Ethics For Data Science Education, Benjamin R. Shapiro, Amanda Meng, Cody O'Donnell, Charlette Lou, Edwin Zhao, Bianca Dankwa, Andrew Hostetler

Learning Sciences Faculty Publications

Data has become central to the technologies and services that human-computer interaction (HCI) designers make, and the ethical use of data in and through these technologies should be given critical attention throughout the design process. However, there is little research on ethics education in computer science that explicitly addresses data ethics. We present and analyze Re-Shape, a method to teach students about the ethical implications of data collection and use. Re-Shape, as part of an educational environment, builds upon the idea of cultivating care and allows students to collect, process, and visualizetheir physical movement data in ways that support critical …

Mission To Planet Markle: Problem-Based Learning For Teaching Elementary Students Difficult Content And Practices, Melanie Peffer, Maggie Renken, Patrick Enderle, Jonathan Cohen

Learning Sciences Faculty Publications

Young children can struggle to learn difficult disciplinary content and important skills for practicing science. Problem-based learning (PBL) may be useful for addressing such difficulties, yet evidence to support its usefulness in elementary school-aged children is limited. We considered the role of a PBL unit in improving students’ genetics content understanding and their skills specific to creating arguments with coordinated claims, evidence, and reasoning. First- through fifth-grade students participated in a six-week PBL unit about evolution and genetics. Students worked in mixed age groups and were charged with illustrating a fictitious alien species, called markles, based on a series of …

Using The Solo Taxonomy To Understand Subgoal Labels Effect In Cs1, Adrienne Decker, Lauren Margulieux, Briana B. Morrison

Learning Sciences Faculty Publications

is work extends previous research on subgoal labeled instructions by examining their effect across a semester-long, Java-based CS1 course. Across four quizzes, students were asked to explain in plain English the process that they would use to solve a programming problem. In this mixed methods study, we used the SOLO taxonomy to categorize student responses about problem-solving processes and compare students who learned with subgoal labels to those who did not. e use of the SOLO taxonomy classification allows us to look deeper than the mere correctness of answers to focus on the quality of the answers produced in terms …

Design And Pilot Testing Of Subgoal Labeled Worked Examples For Five Core Concepts In Cs1, Lauren Margulieux, Briana B. Morrison, Adrienne Decker

Learning Sciences Faculty Publications

Subgoal learning has improved student problem-solving performance in programming, but it has been tested for only one-to-two hours of instruction at a time. Our work pioneers implementing subgoal learning throughout an entire introductory programming course. In this paper we discuss the protocol that we used to identify subgoals for core programming procedures, present the subgoal labels created for the course, and outline the subgoal-labeled instructional materials that were designed for a Java-based course. To examine the effect of subgoal labeled materials on student performance in the course, we compared quiz and exam grades between students who learned using subgoal labels …

Learning Sciences For Computing Education, Lauren Margulieux, Brian Dorn, Kristin Searle

Learning Sciences Faculty Publications

his chapter discusses potential and current overlaps between the learning sciences and computing education research in their origins, theory, and methodology. After an introduction to learning sciences, the chapter describes how both learning sciences and computing education research developed as distinct fields from cognitive science. Despite common roots and common goals, the authors argue that the two fields are less integrated than they should be and recommend theories and methodologies from the learning sciences that could be used more widely in computing education research. The chapter selects for discussion one general learning theory from each of cognition (constructivism), instructional design …

Cognitive Sciences For Computing Education, Anthony V. Robins, Lauren Margulieux, Briana B. Morrison

Learning Sciences Faculty Publications

No abstract provided.

Review Of Measurements Used In Computing Education Research And Suggestions For Increasing Standardization, Lauren Margulieux, Tuba Ketenci, Adriene Decker

Learning Sciences Faculty Publications

The variables that researchers measure and how they measure them are central in any area of research. Which research questions can be asked and how they are answered depends on measurement. This paper describes a systematic review of the literature in computing education research to summarize the commonly used variables and measurements in 197 papers and to compare them to best practices in measurement for human-subjects research. Characteristics of the literature that are examined in the review include variables measured (including learner characteristics), measurements used, and type of data analysis. The review illuminates common practices related to each of these …

Spatial Encoding Strategy Theory: The Relationship Between Spatial Skill And Stem Achievement, Lauren Margulieux

Learning Sciences Faculty Publications

Learners’ spatial skill is a reliable and significant predictor of achievement in STEM, including computing, education. Spatial skill is also malleable, meaning it can be improved through training. Most cognitive skill training improves performance on only a narrow set of similar tasks, but researchers have found ample evidence that spatial training can broadly improve STEM achievement. We do not yet know the cognitive mechanisms that make spatial skill training broadly transferable when other cognitive training is not, but understanding these mechanisms is important for developing training and instruction that consistently benefits learners, especially those starting with low spatial skill. This …

Behavioral Engagement Shifts Among At-Risk High School Students Enrolled In Online Courses, Jennifer Darling-Aduana

Learning Sciences Faculty Publications

Academic behaviors such as attendance are highly associated with academic outcomes. High schools are also increasingly turning to online courses to educate their most marginalized students. In this study, I explored the extent to which enrollment in an online course improved engagement and allowed students to make course progress online outside the traditional school day by examining within-student changes in academic behaviors. Students completed their online course in fewer class periods than required to complete a comparable course in a traditional, face-to-face instructional setting. At the same time, students attended, on average, three additional days of school when enrolled in …

Finding The Best Types Of Guidance For Constructing Self-Explanations Of Subgoals In Programming, Lauren Margulieux, Richard Catrambone

Learning Sciences Faculty Publications

Subgoal learning, a technique used to break down problem solving into manageable pieces, has been used to promote retention and transfer in procedural domains, such as programming. The primary method of learning subgoals has been passive, and passive learning methods are typically less effective than constructive methods. To promote constructive methods of learning subgoals, learners were prompted to self-explain the subgoals of a problem-solving procedure. Self-explanation asks learners to make sense of new information based on prior knowledge and logical reasoning. Self-explanation by novices is typically more effective when they receive guidance, because it helps them to focus on relevant …

Varying Effects Of Subgoal Labeled Expository Text In Programming, Chemistry, And Statistics, Lauren Margulieux, Richard Catrambone, Laura M. Schaeffer

Learning Sciences Faculty Publications

Originally intended as a replication study, this study discusses differences in problem solving performance among different domains caused by the same instructional intervention. The learning sciences acknowledges similarities in the learners’ cognitive architecture that allow interventions to apply across domains, but it also argues that each domain has characteristics that might affect how interventions impact learning. The present study uses an instructional design technique that had previously improved learners’ problem solving performance in programming: subgoal labeled expository text and subgoal labeled worked examples. It intended to replicate this effect for solving problems in statistics and chemistry. However, each of the …

Interaction Geography & The Learning Sciences, Benjamin R. Shapiro

Learning Sciences Faculty Publications

The three papers in this dissertation contribute to research that seeks to characterize the complex and multi-dimensional relation between the physical environment and human learning. The first paper outlines a new approach to describe, represent, and interpret people’s interaction as they move within and across physical environments. I call this approach interaction geography. It encompasses Mondrian Transcription, a method to map people’s movement and conversation over space and time, and the Interaction Geography Slicer (IGS), a dynamic visualization tool that supports new forms of interaction and multi-modal analysis. The second paper extends this work to provide a conceptual framework to …

Personal Curation In A Museum, Benjamin R. Shapiro, Rogers Hall

Learning Sciences Faculty Publications

An established body of work in CSCW and related communities studies social and cooperative interaction in museums and cultural heritage sites. A separate and growing body of research in these same communities is developing ways to understand the design and use of social media from a curating perspective. A curating perspective focuses on how social media is designed and used by people to develop and manage their own digital archives. This paper uses a cultural heritage museum as the empirical basis and setting along with new information visualization methods we have developed to better integrate these bodies of work and …

Preservice And Early Career Teachers’ Preconceptions And Misconceptions About Making In Education, Jonathan Cohen, W. Monty Jones, Shaunna Smith

Learning Sciences Faculty Publications

This qualitative study examined preservice and early career teachers’ preconceptions and misconceptions about making in education. Eighty-two preservice and early career teachers participated in brief, one-time maker workshops, then wrote reflections on their experiences. Using constant comparative analysis, researchers uncovered two common misconceptions held by the participants. The first was that making in education consisted of hands-on activities designed to achieve specific content learning objectives. The second was that making was largely dependent on the use of advanced manufacturing tools, such as 3D printers. Such misconceptions could negatively impact the potential of making in education. Recommendations for resolving these misconceptions …