Educational Assessment, Evaluation, and Research Commons^™

Narrative World Building: Creative Applications For Gamification In Study Abroad, Ashley Lear

Publications

This study examined a cohort of 12 study abroad participants taking a course on video game topography and narrative in Salamanca, Spain, to determine how inhabiting and co-creating narrative worlds as part of the coursework might impact the experiences of the students inside and outside of the classroom as they engaged in mandated and optional cultural engagement activities, such as museum tours and excursions to historical sites. Students completed two gameful learning activities: 1) they co-created their own narrative game world in a group game proposal assignment drawing upon research from storytelling through game environments, and 2) they created independent …

Identifying Support Structures Associated With Informal Formative Evaluation In Instructional Design, Suzanne Smith, Tian Luo

STEMPS Faculty Publications

Instructional designers perform informal formative evaluation in design practice. An evaluation may be used to locate errors in alignment of instructional objectives or to increase the quality or effectiveness of a design. An instructional design review is similar to peer reviews in higher education which are often structured, and tools are provided to contribute to the review. A study was performed to identify the support structures and tools that contribute to building a community of feedback within the practice of instructional design reviews. Six instructional designers and design managers were interviewed to gather the processes they use in design reviews …

Starting The Journey To Excellence With A College Faculty Onboarding Program, Robin Cooper, Judith Slapak-Barski

HCAS Instructional Design and Pedagogy

The journey to engaged, excellent teaching requires support and preparation for those doing the teaching. In this session, we describe a college Faculty Onboarding Program we developed in 2023 to introduce our new Halmos faculty to NSU (Nova Southeastern University) and to Halmos, and to provide them with the resources and information needed to succeed in their new role—driven by the belief that effective onboarding will lead to higher job satisfaction, better job performance, greater organizational commitment, and reduction in stress. Informed by feedback from recently hired faculty, we identified support strategies to help new faculty thrive by proactively removing …

Proceedings Of The Cuny Games Conference 9.0, Robert O. Duncan, Grace L. Axler-Diperte, Joseph Bisz, Christina Boyle, Devorah Kletenik, Carolyn Stallard

Publications and Research

The CUNY Games Network is an organization dedicated to encouraging research, scholarship and teaching in the developing field of games-based learning. We connect educators from every campus and discipline at CUNY and beyond who are interested in digital and non-digital games, simulations, and other forms of interactive teaching and inquiry-based learning.

One Font Doesn’T Fit All: The Influence Of Digital Text Personalization On Comprehension In Child And Adolescent Readers, Shannon M. Sheppard, Susanne L. Nobles, Anton Palma, Sophie Kajfez, Marjorie Jordan, Kathy Crowley, Sofie Beier

Communication Sciences and Disorders Faculty Articles and Research

Reading comprehension is an essential skill. It is unclear whether and to what degree typography and font personalization may impact reading comprehension in younger readers. With advancements in technology, it is now feasible to personalize digital reading formats in general technology tools, but this feature is not yet available for many educational tools. The current study aimed to investigate the effect of character width and inter-letter spacing on reading speed and comprehension. We enrolled 94 children (kindergarten–8th grade) and compared performance with six font variations on a word-level semantic decision task (Experiment 1) and a passage-level comprehension task (Experiment 2). …

How Teachers Conceptualise Shared Control With An Ai Co-Orchestration Tool: A Multiyear Teacher-Centred Design Process, Luettamae Lawrence, Vanessa Echeverria, Kexin Yang, Vincent Aleven, Nikol Rummel

Instructional Technology and Learning Sciences Faculty Publications

Artificial intelligence (AI) can enhance teachers' capabilities by sharing control over different parts of learning activities. This is especially true for complex learning activities, such as dynamic learning transitions where students move between individual and collaborative learning in un-planned ways, as the need arises. Yet, few initiatives have emerged considering how shared responsibility between teachers and AI can support learning and how teachers' voices might be included to inform design decisions. The goal of our article is twofold. First, we describe a secondary analysis of our co-design process comprising six design methods to understand how teachers conceptualise sharing control with …

How Co-Designing Computational Modeling Activities Helped Teachers Implement Responsive Teaching Strategies, Hillary Swanson, Luettamae Lawrence, Jared Arnell, Allisia Dawkins, Bonni Jones, Bruce Sherin, Uri Wilensky

Instructional Technology and Learning Sciences Faculty Publications

In recent years, science education has shifted focus, from content to practice. This is reflected in the NGSS, which advocate learning science concepts through engagement in science and engineering practices. Theory building is a central activity of science and computational modeling is a key practice through which contemporary scientists construct theory. In this paper, we discuss an 8th grade science teacher's implementation of a computational modeling lesson. The teacher had co-designed the computational modeling microworld and lesson with the research team over the preceding summers. We investigate the teacher's activity during a whole-class discussion near the end of the lesson, …

Understanding Computational Thinking In The Gameplay Of The African Songo Board Game, Rebecca Y. Bayeck

Instructional Technology and Learning Sciences Faculty Publications

Computational thinking is a necessary skill for the 21st century. While previously examined in computer-rich settings, researchers are increasingly studying computational thinking in unplugged environments such as board games. Focusing on the African board game Songo, this study shows that computational thinking practices are embedded in Songo board gameplay and interact with the cultural context. The study also reveals a computing practice peculiar to Songo gameplay, songoputation. This paper suggests that researchers can benefit from exploring computational thinking and computing practices beyond board games in western contexts.

Supporting And Sustaining Equitable Steam Activities In High School Classrooms: Understanding Computer Science Teachers’ Needs And Practices When Implementing An E-Textiles Curriculum To Forge Connections Across Communities, Deborah A. Fields, Yasmin B. Kafai

Instructional Technology and Learning Sciences Faculty Publications

While the last two decades have seen an increased interest in STEAM (science, technology, engineering, arts, and mathematics) in K-12 schools, few efforts have focused on the teachers and teaching practices necessary to support these interventions. Even fewer have considered the important work that teachers carry out not just inside classrooms but beyond the classroom walls to sustain such STEAM implementation efforts, from interacting with administrators to recruiting students and persuading parents about the importance of arts and computer science. In order to understand teachers’ needs and practices regarding STEAM implementation, in this paper, we focus on eight experienced computer …

Is Microethnography An Ethnographic Case Study? And/Or A Mini-Ethnographic Case Study? An Analysis Of The Literature, Rebecca Y. Bayeck

Instructional Technology and Learning Sciences Faculty Publications

Selecting the research approach that addresses the research question is often challenging for novice researchers. However, getting a better understanding of the research approaches available in the field, is likely to help novice researchers identify and choose the research approach that fits their situation. In this paper, we discuss microethnography, ethnographic case study, and mini-ethnography case study in order to show that these approaches may have similarities but are different. The author hopes that this discussion will help researchers get a better understanding of these approaches and dissipate the confusion that may exist.

The Predictive Validity Of The Gre Across Graduate Outcomes: A Meta-Analysis Of Trends Over Time, David F. Feldon, Kaylee Litson, Brinleigh Cahoon, Zhang Feng, Andrew Walker, Colby Tofel-Grehl

Instructional Technology and Learning Sciences Faculty Publications

This meta-analysis assesses the predictive validity of the Graduate Record Examination (GRE) across outcome variables, including grade point average, for graduate students. In addition to aggregate effects, this paper also assessed changes in observed effects over time as related to increasing diversity in the graduate student population and as a function of gender and racial/ethnic composition of study samples. Framed using a lens of critical whiteness, this analysis examined n = 1,744 individual effects across k = 208 studies. Overall, 61.6% of reported effects were nonsignificant (i.e. no predictive value of GRE scores on student outcomes). Further, the magnitude of …

Thinking Global, Acting Local, Ali Gohar Qazi

Institute for Educational Development, Karachi

Ongoing or continuous professional development is essential for teachers to develop and maintain the knowledge base required to produce more powerful learning outcomes among their students. Research has shown that professional development is more effective and meaningful to teachers when it is content focused, involves active learning and collective participation, and when it is sustained in duration, instead of being top-down, episodic, or delivered as ‘one-shot’ training workshops.

Designing Hybrid Human-Ai Orchestration Tools For Individual And Collaborative Activities: A Technology Probe Study, Vanessa Echeverria, Kexin Yang, Luettamae Lawrence, Nikol Rummel, Vincent Aleven

Instructional Technology and Learning Sciences Faculty Publications

Combining individual and collaborative learning is common, but dynamic combinations (which happen as-the-need arises, rather than in pre-planned ways, and may happen on an individual basis) are rare. This work reports findings from a technology probe study exploring alternative designs for classroom co-orchestration support for dynamically transitioning between individual and collaborative learning. The study involved 1) a technology-probe classroom study in an authentic, AI-supported classroom to understand teachers' and students' needs for co-orchestration support over dynamic transitions; and 2) workshops and interviews with students and teachers to get informed feedback about their lived experiences. 118 students and three teachers from …

Online Early Childhood Teacher Education Program: Practices And Future Possibilities, Yasmeen Mehboob, Silpa Aziz Karim

Institute for Educational Development, Karachi

The study aimed to test the effectiveness and the feasibility of Zoom and Virtual Learning Environments (VLE) for early childhood teacher education (ECTE) at a university in Pakistan. The study addresses how a Community of Inquiry (COI) framework can be embedded within Zoom and VLE to make the online course meaningful, interactive and accessible. This is an exploratory case study with a mixed method approach. Data was gathered through survey, focus group discussion, reflective journals, observations and meeting notes. Pre- and post-online knowledge tests and assignments were used to assess changes in participant’s knowledge and understanding. Survey result indicated positive …

Authentic Learning Design Failures: The Need For Learner And Contextual Analysis And Participatory Design, Mohan Yang, Jon Harbor

STEMPS Faculty Publications

This design case examines what program leaders learned from failures in the design of a program of authentic learning about teaching diverse audiences through educational outreach. The program was initiated and then redesigned to develop the teaching and communication skills of graduate students from a wide range of backgrounds by engaging them in authentic experiences with middle school teachers and students. Analysis of post-program data revealed seven design failures related to the lack of upfront analysis to inform the program design. Each design failure was detailed through a fishbone diagram method to identify the corre- sponding contributing factors. The failures …

"The Village That Learns": A Learning Journey Across Intraventions And Domains Over Two Decades In A Rural Thai Community, Deborah A. Fields, Luis Morales-Navarro, Paulo Blikstein

Instructional Technology and Learning Sciences Faculty Publications

What does it mean to become a village that learns? In this paper we document the transformative learning journey of a small Thai village over 24 years, becoming a community that identified, tackled, and iterated on problems, altering their everyday practices and lives. In that process the village shifted from a subsistence agricultural community staggeringly in debt to one known for its sustainable environmental, agricultural, and financial initiatives. To understand the village’s learning journey, we consider the village itself as the primary unit of analysis, applying an iterative case study approach, with chronological sequencing, thematic, and biographical narrative analysis …

A Guided Chatbot Learning Experience In The Science Classroom, Jennifer Davis

Department of Teaching, Learning, and Teacher Education: Dissertations, Theses, and Student Research

This dissertation describes a practitioner’s design-based development of a prototype chatbot to guide students in learning biological concepts of genetic mutations and protein synthesis. This chatbot’s architecture provides learning activities, feedback, and support throughout a series of short, connected lessons. The chatbot is designed to scaffold learners through a predict, observe, explain model of inquiry learning. It utilizes real-world phenomena to lead students through biology core ideas, science and engineering practices, and crosscutting concepts. Results of prototype testing include survey results in support of the proof of concept among both students and teachers, as well as accuracy measurements of chatbot …

Positionality: The Interplay Of Space, Context And Identity, Rebecca Y. Bayeck

Instructional Technology and Learning Sciences Faculty Publications

This article considers the way in which positionality shifts and is formed during a cross-cultural study to reveal the complexity of the insider-outsider status. As a researcher in a male-dominated game setting, I reflect on the research process and my interactions with participants to show the interplay of space, context, and identity in shaping a researcher’s status. I discuss the process of gaining access to the research site and participants, and data collection in relation to space, context, and identity. The interaction of my identities with space, and context informed my status at various moments. This interplay constructs a complex …

Exploring The Level Of Engagement Of Capstone Students In An Active Learning Center, Carol Ann Sharicz

Instructional Design Faculty Publications

This study provides a qualitative exploration of the level of engagement of Capstone students attending course meetings in an Active Learning Center (ALC) during their own respective semester at a public research university in the Boston area. A design-based research methodology was employed to study innovative learning environments encompassing learning in context. Study results uncovered three themes regarding the impact of the learning space on students’ perceptions of learning and levels of engagement; revealed that one particular tool, the use of small white boards, encouraged deep thinking and reflection; and indicated that the ALC provided a highly collaborative experience.

The Intersection Of Cultural Context And Research Encounter: Focus On Interviewing In Qualitative Research, Rebecca Yvonne Bayeck

Instructional Technology and Learning Sciences Faculty Publications

This article discusses the influence of the cultural context on the interview process. With literature demonstrating the role of spatial context on interviews, the article contends that similar consideration should be given to cultural contexts of research studies. Focusing on the cultural context where the interview takes place and the interactions during the interview can help researchers understand and analyze interview material. Interview forms such as conversation/interview bombing emerged from the interaction of cultural context with the interview process. This points to the need for qualitative researchers to explore how the cultural context shapes their research encounter. Such focus will …

Analysis Of Students’ Multi-Representation Ability In Augmented Reality-Assisted Learning, Sri Jumini, Edy Cahyono, Muhamad Miftakhul Falah

Library Philosophy and Practice (e-journal)

Not all learning sources can directly and cheaply be presented, so augmented reality media is needed to be applied to students with various talents and intelligence. This study aims to analyze students’ multi-representation ability through the use of augmented reality media. The research method was carried out through pre-experiment with one group posttest only design. Test question items were given to see the students’ multi-representation ability. Data analysis was carried out through the percentage of the number of students achieving test scores of more than or equal to 80 on a scale of 100. The results showed that 88% (28 …

Understanding The Transient Nature Of Stem Doctoral Students’ Research Self-Efficacy Across Time: Considering The Role Of Gender, Race, And First-Generation College Status, Kaylee Litson, Jennifer M. Blaney, David F. Feldon

Instructional Technology and Learning Sciences Faculty Publications

Developing research self-efficacy is an important part of doctoral student preparation. Despite the documented importance of research self-efficacy, little is known about the progression of doctoral students’ research self-efficacy over time in general and for students from minoritized groups. This study examined both within- and between-person stability of research self-efficacy from semester to semester over 4 years, focusing on doctoral students in biological sciences (N = 336). Using random intercept autoregressive analyses, we evaluated differences in stability across gender, racially minoritized student status, and first-generation student status. Results showed similar mean levels of self-efficacy across demographic groups and across …

Communicating About Computational Thinking: Understanding Affordances Of Portfolios For Assessing High School Students’ Computational Thinking And Participation Practices, Deborah A. Fields, Debora Lui, Yasmin Kafai, Gayithri Jayathirtha, Justice Walker, Mia Shaw

Instructional Technology and Learning Sciences Faculty Publications

Background and Context: While assessment of computational thinking concepts, practices, and perspectives is at the forefront of K-12 CS education, supporting student communication about computation has received relatively little attention.

Objective: To examine the usability of process-based portfolios for capturing students’ communication about their computational practices regarding the process of making electronic textile projects.

Method: We examined the portfolios of 248 high school students in 15 introductory CS classrooms from largely underserved communities, using a formal rubric (top-down) to code computational communication and an open-coding scheme (bottom-up) to identify computational practices described.

Findings: Students demonstrated stronger …

Debugging By Design: A Constructionist Approach To High School Students’ Crafting And Coding Of Electronic Textiles As Failure Artifacts, Deborah A. Fields, Yasmin B. Kafai, Luis Morales-Navarro, Justice T. Walker

Instructional Technology and Learning Sciences Faculty Publications

Much attention in constructionism has focused on designing tools and activities that support learners in designing fully finished and functional applications and artifacts to be shared with others. But helping students learn to debug their applications often takes on a surprisingly more instructionist stance by giving them checklists, teaching them strategies or providing them with test programs. The idea of designing bugs for learning—or debugging by design—makes learners agents of their own learning and, more importantly, of making and solving mistakes. In this paper, we report on our implementation of “Debugging by Design” activities in a high school classroom over …

When Knowing Is Not Enough: A Narrative Exploration Of How K-12 Teachers Make Decisions About The Transfer Of Critical Competencies From Professional Learning To Daily Practice, Nell E. Ballard-Jones

Antioch University Full-Text Dissertations & Theses

School districts spend millions of dollars each year to provide training and learning to staff working in direct and indirect service to students (National Council on Teacher Quality, 2021). This financial commitment says nothing about what is even more important: the need for school employees and the systems in which we work to serve students more effectively. Despite vast allocations of time and money and presumably best intentions for better social and academic outcomes for students, very little data exist that reflect regular transfer and application of training/learning into professional practice (Nittler et al., 2015). By and large, schools and …

Leveraging Local Resources And Contexts For Inclusive Computer Science Classrooms: Reflections From Experienced High School Teachers Implementing Electronic Textiles, Mia S. Shaw, Deborah A. Fields, Yasmin B. Kafai

Instructional Technology and Learning Sciences Faculty Publications

Background and context

Promoting open-ended projects presents new opportunities and challenges for inclusive teaching in CS classrooms. While efforts have been made to develop inclusive curricula, little research has focused on ways teachers apply curricula in their classrooms to promote inclusion.

Objective

To understand the challenges faced in facilitating an open-ended unit and the pedagogical strategies enacted to address those challenges, we analyze the self-reported teaching practices that experienced teachers developed in their implementation of a constructionist electronic textiles unit in Exploring Computer Science.

Method

We inductively analyzed and coded 17 experienced teachers’ weekly surveys and post-interviews.

Findings

Teachers …

Designing Analog Learning Games: Genre Affordances, Limitations And Multi-Game Approaches, Owen Gottlieb, Ian Schreiber

Articles

This chapter explores what the authors discovered about analog games and game design during the many iterative processes that have led to the Lost & Found series, and how they found certain constraints and affordances (that which an artifact assists, promotes or allows) provided by the boardgame genre. Some findings were counter-intuitive. What choices would allow for the modeling of complex systems, such as legal and economic systems? What choices would allow for gameplay within the time of a class-period? What mechanics could promote discussions of tradeoff decisions? If players are expending too much cognition on arithmetic strategizing, could that …

Faculty Perceptions Of Online Teaching At A Midsized Liberal Arts University, Dana L. Shreaves, Yu-Hui Ching, Lida Uribe-Florez, Jesús Trespalacios

Educational Technology Faculty Publications and Presentations

In this mixed-methods study, faculty perceptions of online teaching at a midsized liberal arts university were examined to better understand faculty acceptance and participation in online teaching at the university. Seventy-nine participants responded to a survey that collected qualitative and quantitative data. Content analysis of faculty perceptions of online teaching was employed and resulted in the identification of six themes. An examination of 21 quantitative factors identified 17 factors reported by more than 50% of respondents to influence their decision to teach or not teach online. Study participants perceived online learning as attractive to students but they wanted any online …

Finding A Fit: Biological Science Doctoral Students’ Selection Of A Principal Investigator And Research Laboratory, Michelle A. Maher, Annie M. Wofford, Josipa Roksa, David F. Feldon

Instructional Technology and Learning Sciences Faculty Publications

In the laboratory-based disciplines, selection of a principal investigator (PI) and research laboratory (lab) indelibly shapes doctoral students’ experiences and educational outcomes. Framed by the theoretical concept of person–environment fit from within a socialization model, we use an inductive, qualitative approach to explore how a sample of 42 early-stage doctoral students enrolled in biological sciences programs made decisions about fitting with a PI and within a lab. Results illuminated a complex array of factors that students considered in selecting a PI, including PI relationship, mentoring style, and professional stability. Further, with regard to students’ lab selection, peers and research projects …

“How Well Does Your Structural Equation Model Fit Your Data?”: Is Marcoulides And Yuan’S Equivalence Test The Answer?, James Peugh, David F. Feldon

Instructional Technology and Learning Sciences Faculty Publications

Structural equation modeling is an ideal data analytical tool for testing complex relationships among many analytical variables. It can simultaneously test multiple mediating and moderating relationships, estimate latent variables on the basis of related measures, and address practical issues such as nonnormality and missing data. To test the extent to which a hypothesized model provides an appropriate characterization of the collective relationships among its variables, researchers must assess the “fit” between the model and the sample’s data. However, interpreting estimates of model fit is a problematic process. The traditional inferential test of model fit, the chi-square test, is biased due …