Engineering Education Commons^™

Examining Data Richness In Undergraduate Students’ Reflections: A Linguistic Analysis Of Three Data Collection Methods, Kirsten A. Davis, Anne Wrobetz

School of Engineering Education Faculty Publications

Researchers select data collection methods for a variety of reasons. In our research on experiential education programs, we perceived that when we collected students’ reflections using different methods, the data we received was different. However, we found few existing studies to help us characterize what was different across our data sets to inform our decisions about which methods to use. Thus, the purpose of our study was to compare data richness across three data sets from previous studies of engineering students studying abroad, each collected using a different method (written diaries, interviews, video diaries). We used linguistic inquiry as a …

Developing Indices To Study Engineers' Interest, Confidence, And Experience With Global Engineering Work Activities, Lexy Arinze, Joe Tort, Kirsten A. Davis

School of Engineering Education Faculty Publications

Background

As engineering work has become increasingly globalized, prior studies have examined the intercultural attitudes and competencies of engineering students and practicing engineers. However, we lack tools to study how engineers engage in global work across industries, roles, and organizational contexts.

Purpose

The purpose of this study was to develop a tool to measure engineers' interest, confidence, and frequency in performing global work tasks. To this end, we designed the Global Task Performance (GTP) indices, which evaluate four dimensions of engagement in global engineering work.

Methods

Guided by Social Cognitive Career Theory and the Global Engineering Competency (GEC) framework, we …

Developing Students' Sociotechnical Thinking In The Humanities-Engineering Classroom: A Treatment Versus Control Study Using A Scenario-Based Assessment, Lori Czerwionka, Siddhant S. Joshi, Gabriel O. Rios-Rojas, Kirsten A. Davis

School of Engineering Education Faculty Publications

Background

Engineering problems are open-ended and complex, involving technical and social aspects, yet engineering education focuses on technical training and closed-ended problems. To prepare engineering students, curricula should foster sociotechnical thinking—the ability to consider the interplay of technical and social factors during problem solving. Although educational interventions in this area show promise, further insight into the benefits of interdisciplinary interventions is warranted, particularly with treatment and control group designs.

Purpose

We examine the effect of a Humanities-Informed Engineering Projects (HIEP) course on students' sociotechnical thinking development.

Method

We compared the sociotechnical thinking development of engineering students who took the HIEP …

Exploring How Personal And Program Characteristics Inform The Experiences Of Engineering Students Abroad, Kirsten A. Davis, David B. Knight

School of Engineering Education Faculty Publications

Background

As more universities seek to offer international experiences for engineering students, it is important to design such programs to effectively support student learning abroad. Previous research on study abroad has focused on a limited number of outcomes and therefore failed to consider the diversity of experiences students may have in the same program and the range of learning outcomes they may develop while abroad.

Purpose

We explored student experiences across multiple types of engineering study abroad programs to address the research question: How do the types of significant experiences students highlighted from their time abroad differ based on student …

Fostering Systems Thinking Through Engineering Study Abroad Programs, Desen S. Ozkan, Kirsten A. Davis, James C. Davis, Jessica Deters, Homero Murzi

School of Engineering Education Faculty Publications

Engineering study abroad programs are a novel setting for students to develop systems thinking, which has primarily been taught in formal learning environments. Through a situated learning perspective, we examined the impact of two engineering-focused study abroad experiences on developing students’ systems thinking skills. We conducted secondary data analysis to examine how first-year engineering students (n = 47) at a technical university in the United States (US) connect technical and contextual aspects of engineering in China and the UK/Ireland. The students on the China track drew connections more frequently (81% of excerpts) than those on the UK/Ireland track (49% …

Ai-Enhanced Decision-Making For Course Modality Preferences In Higher Engineering Education During The Post-Covid-19 Era, Amirreza Mehrabi, Jason Morphew, Babak Nadjar Araabi, Negar Memarian, Hossein Memarian

School of Engineering Education Faculty Publications

The onset of the COVID-19 pandemic has compelled a swift transformation in higher-education methodologies, particularly in the domain of course modality. This study highlights the potential for artificial intelligence and machine learning to improve decision-making in advanced engineering education. We focus on the potential for large existing datasets to align institutional decisions with student and faculty preferences in the face of rapid changes in instructional approaches prompted by the COVID-19 pandemic. To ascertain the preferences of students and instructors regarding class modalities across various courses, we utilized the Cognitive Process-Embedded Systems and e-learning conceptual framework. This framework effectively delineates the …

Mechanics Cognitive Diagnostic: Mathematics Skills Tested In Introductory Physics Courses, Vy Le, Ben Van Dusen, Jayson M. Nissen, Xiuxiu Tang, Yuxiao Zhang, Hua Hua Chang, Jason Morphew

School of Engineering Education Faculty Publications

Physics instructors and education researchers use research-based assessments (RBAs) to evaluate students' preparation for physics courses. This preparation can cover a wide range of constructs including mathematics and physics content. Using separate mathematics and physics RBAs consumes course time. We are developing a new RBA for introductory mechanics as an online test using both computerized adaptive testing and cognitive diagnostic models. This design allows the adaptive RBA to assess mathematics and physics content knowledge within a single assessment. In this article, we used an evidence-centered design framework to inform the extent to which our models of skills students develop in …

Characterising Stem Ways Of Thinking In Engineering Design (Ed)-Based Tasks, Ravishankar Chatta Subramaniam, Caden L’Fontaine, Amir Bralin, Jason Morphew, Carina M. Rebello, Sanjay Rebello

School of Engineering Education Faculty Publications

Investigating students' thinking in routine classroom tasks, especially in science and engineering, is crucial. Given the rising interest in STEM Ways of Thinking (SWoT), in this exploratory study, we focus on two multi-week Engineering Design tasks within an undergraduate physics laboratory. Given that the term 'ways of thinking' has varied interpretations, we aim to further the discourse by identifying four SWoTs: Design Thinking, Physics Concepts, Mathematical Constructs, and Metacognitive Reflection. Analyzing discussions from 14 student-groups reveals notable differences in how students solve an instructor-assigned challenge given earlier in the semester and a student-generated challenge later in the semester. Students considered …

Ai-Based Concept Inventories: Using Cognitive Diagnostic Computer Adaptivetesting In Lasso For Classroom Assessment, Jason Morphew, Amirreza Mehrabi, Ben Van Dusen, Jayson Nissen

School of Engineering Education Faculty Publications

Research-based concept inventories, such as the Force Concept Inventory, the Force and Motion Conceptual Evaluation, or the Energy and Momentum Conceptual Survey have played a pivotal role in designing and evaluating instruction in introductory physics courses. While these concept inventories have helped identify inequity in course instruction and have led to improved pedagogical methods, concerns about their ability to provide actionable feedback for formative assessment remain. In addition, using the same questions for both a pre- and a post- assessment leaves the potential for students learning the correct answers without undergoing conceptual change. To address these issues, we designed a …

A Novel Research Design: Using Multilevel Discrete-Time Survival Analysis To Investigate The Effect Of Calculus I On Engineering Student Persistence, Hayaam Osman, Matthew Ohland

School of Engineering Education Faculty Publications

The persistence of engineering students through graduation continues to be a concern in higher education. Previous studies have highlighted a link between students' performance in introductory mathematics courses and graduation rates. Focusing on a crucial foundational course within the engineering curriculum, the purpose of this study is to investigate how students’ performance in Calculus I impact their persistence in the engineering program. Utilizing data from 22 diverse educational institutions using Multiple-Institution Database for Investigating Engineering Longitudinal Development (MIDFIELD), multilevel discrete-time survival analysis is employed to examine the longitudinal persistence patterns among the nested structure of students within institutions. Discrete-time analysis …

A Design Team’S Perspective-Mixing For Shared Empathic Understandings, Eunhye Kim, Robin Adams, Senay Purzer

School of Engineering Education Faculty Publications

Empathic design involves two social practices – collaboration with users and intra team collaboration forshared empathic understandings. This study is focused on the latter phenomenon, aiming to characterise a social mechanism underlying it. To achieve this objective, we used a framework of perspective-mixing in empathic design, including three distinct perspectives: first, second, and third-person perspectives. With this framework, we traced how team members share and integrate individuals’ user knowledge and insights to develop and apply their shared empathic understandings. We conducted conversation analysis to examine one design team’s conversations over a design journey from need-finding to initial ideation to prototyping …

Towards A Distributed Model Of Teaming: Instructor-Driven Lessons From I-Matter, Austin Morgan Kainoa Peters, Alice Pawley, Stephanie Masta, Darryl Dickerson, Matthew Ohland

School of Engineering Education Faculty Publications

This WIP research paper describes the development of a preliminary practical model to improve how instructors of large classes can address marginalization amongst teammates in small teams. We collected interviews with instructors of large first-year engineering classes at a large Midwestern, primarily white, research-intensive university, and analyzed them thematically drawing on Sue and colleagues’ microaggression theory, Cortina and colleagues’ articulation of selective incivility, and theory on coded language. Through iterative readings and coding, we offer an initial articulation of attributes demonstrated by instructors, and position those attributes in a preliminary contrasting model that prioritizes distributed responsibility over instructors being solely …

Board 398: Sustaining And Scaling The Impact Of The Midfield Project At The American Society For Engineering Education (Year 2), Susan M. Lord, Matthew Ohland, Marisa K. Orr, Richard A. Layton, Catherine E. Brawner, Russell Long, Haleh Barmaki Brotherton, Hayaam Osman, Joe Roy

School of Engineering Education Faculty Publications

The Multiple Institution Database for Investigating Engineering Longitudinal Development (MIDFIELD) has been developed over many years with substantial investment by the National Science Foundation through Engineering Education and Centers in the Engineering Directorate and the Division of Undergraduate Education in the Education and Human Resources Directorate. This project is focused on transitioning MIDFIELD to the American Society for Engineering Education (ASEE). The current team of MIDFIELD researchers continues to support this project including helping others learn to use the database. We have developed detailed tutorials in R that introduce MIDFIELD, key metrics, and example scenarios. We have also designed and …

Board 438: Year Two Of Developing A New Dataset For Analyzing Engineering Curricula, David Reeping, Kenneth Reid, Matthew Ohland, Nahal Rashedi

School of Engineering Education Faculty Publications

This paper discusses the developments during Year 2 for a project concerned with analyzing the curricula of engineering programs in the United States to understand the structural barriers embedded in degree requirements that could push out diverse groups of students. We are using an emerging method for quantifying the complexity of these programs called Curricular Analytics. This method involves treating the prerequisite relationships between courses as a network and applying graph theoretic measures to calculate a curriculum’s structure complexity. In Year 1, we collected 497 plans of study representing five engineering disciplines (i.e., Mechanical, Civil, Electrical, Chemical, and Industrial) across …

Course Experiences That Promote And Inhibit Human-Centered Design, Elizabeth A. Sanders, Molly Goldstein, Justin L. Hess

School of Engineering Education Faculty Publications

Engineers contribute to large-scale socio-technical challenges, and human-centered design offers a design thinking approach that helps engineers develop a thorough understanding of the socio-technical effects of their design work. Thus, effective strategies for assessing and teaching human-centered design are needed. This study aimed to identify course characteristics that influence how students experience human-centered design in an introductory systems engineering design course. First, we categorized open-ended written reflections to understand the degree to which students experienced human-centered design. Second, we performed a thematic analysis to characterize salient course experiences for two groups of students: (1) students who experienced human-centered design in …

Exploring Scenario-Based Assessment Of Students' Global Engineering Competency: Building Evidence Of Validity Of A China-Based Situational Judgment Test, Kirsten A. Davis, Brent Jesiek, David B. Knight

School of Engineering Education Faculty Publications

Background

Engineers operate in an increasingly global environment, making it important that engineering students develop global engineering competency to prepare them for success in the workplace. To understand this learning, we need assessment approaches that go beyond traditional self-report surveys. A previous study (Jesiek et al., Journal of Engineering Education 2020; 109(3):1–21) began this process by developing a situational judgment test (SJT) to assess global engineering competency based in the Chinese context and administering it to practicing engineers.

Purpose

We built on this previous study by administering the SJT to engineering students to explore what prior experiences related to their …

A Primer On Working With Longitudinal Student Unit Records, Russell Long, Richard A. Layton, Marisa K. Orr, Susan M. Lord, Matthew Ohland

School of Engineering Education Faculty Publications

Longitudinal, student-level data are a rich resource for characterizing how students navigate the terrain of higher education. Learning to work effectively with such data, however, can be a challenge. In this paper, we share some of our experiences over years of conducting research with the Multiple Institution Database for Investigating Engineering Longitudinal Development (MIDFIELD). MIDFIELD contains individual student-level records for all undergraduate students at 19 US institutions with over 1.7 million unique students. This paper focuses on our lessons learned about processing longitudinal data to prepare it for analysis. We describe and define the steps that we take to process …

Comparing Self-Report Assessments And Scenario-Based Assessments Of Systems Thinking Competence, Kirsten A. Davis, Dustin Grote, Hesam Mahmoudi, Logan Perry, Navid Ghaffarzadegan, Jacob Grohs, Niyousha Hosseinichimeh, David B. Knight, Konstantinos Triantis

School of Engineering Education Faculty Publications

Self-report assessments are used frequently in higher education to assess a variety of constructs, including attitudes, opinions, knowledge, and competence. Systems thinking is an example of one competence often measured using self-report assessments where individuals answer several questions about their perceptions of their own skills, habits, or daily decisions. In this study, we define systems thinking as the ability to see the world as a complex interconnected system where different parts can influence each other, and the interrelationships determine system outcomes. An alternative, less-common, assessment approach is to measure skills directly by providing a scenario about an unstructured problem and …

Exploring Narratives Of Researcher Development For Student Researchers Abroad, Kirsten A. Davis, Teirra Holloman, Jessica Deters, David B. Knight

School of Engineering Education Faculty Publications

Background: Research experiences for undergraduate and graduate students have the potential to provide significant learning experiences, but most research of these experiences focuses primarily on their outcomes. To better design such experiences for a diverse range of students, it is important to understand student narratives of researcher development and identify significant experiences in their development as researchers.

Purpose: In this study, we explored narratives of researcher development for seven US civil engineering students in an eight-week research experience in Australia. Through these narratives, we sought to identify categories of significant experiences to characterize the early stage of the researcher development …

Board 201: A New Public Dataset For Exploring Engineering Longitudinal Development By Leveraging Curricular Analytics, David Reeping, Matthew Ohland, Kenneth Reid, Hossein Ebrahiminejad, Nahal Rashedi

School of Engineering Education Faculty Publications

Considering the increasing demand for engineering graduates, understanding what is limiting students from completing their degrees has been a consistent question posed in the literature. The nontrivial variance in pathways students take in obtaining an engineering degree, especially in cases where students abandon their studies, suggests that longitudinal datasets can hold a wealth of information to uncover factors contributing to attrition. Accordingly, this project uses existing data to explore curricular factors that create barriers for different students by leveraging a new framework for quantifying the impact of such factors, Curricular Analytics. Curricular Analytics uses network analysis to measure sequencing and …

Board 394: Sustaining And Scaling The Impact Of The Midfield Project At The American Society For Engineering Education (Year 1), Susan M. Lord, Matthew Ohland, Marisa K. Orr, Russell Long, Catherine E. Brawner, Joseph Roy

School of Engineering Education Faculty Publications

A substantial investment by the National Science Foundation (NSF), including awards from Engineering Education and Centers in the Engineering Directorate and the Division of Undergraduate Education in the Education and Human Resources Directorate, has led to the creation and study of the Multiple Institution Database for Investigating Engineering Longitudinal Development (MIDFIELD). This large database of student records has yielded groundbreaking research on student pathways by a small interdisciplinary team of researchers. The team has shown that while individual engineering programs may have poor graduation rates, a multi-institutional view reveals that engineering programs as a whole graduate a larger fraction of …

A Qualitative Study Of Emotions Experienced By First-Year Engineering Students During Programming Tasks, Zahra Atiq, Michael C. Loui

School of Engineering Education Faculty Publications

In introductory computer programming courses, students experience a range of emotions. Students often experience anxiety and frustration when they encounter difficulties in writing programs. Continued frustration can discourage students from pursuing engineering and computing careers. Although prior research has shown how emotions affect students’ motivation and learning, little is known about students’ emotions in programming courses. In this qualitative study of first-year engineering students taking an introductory programming course, we examined the emotions that these students experienced during programming tasks and the reasons for experiencing those emotions. Our study was grounded in the control-value theory of achievement emotions. Each research …

Assessing Learning Processes Rather Than Outcomes: Using Critical Incidents To Explore Student Learning Abroad, Kirsten A. Davis, David B. Knight

School of Engineering Education Faculty Publications

There is an increasing emphasis on assessing student learning outcomes from study abroad experiences, but this assessment often focuses on a limited range of outcomes and assessment methods. We argue for shifting to assessing student learning processes in study abroad and present the critical incident technique as one approach to achieve this goal. We demonstrate this approach in interviews with 79 students across a range of global engineering programs, through which we identified 173 incidents which were analyzed to identify common themes. This analysis revealed that students described a wide range of experiences and outcomes from their time abroad. Students’ …

Research Experiences Instrument Scoring Guide, Eric Holloway, Kerrie Anna Douglas, William Charles Oakes, David Radcliffe

School of Engineering Education Faculty Publications

This document is a scoring guide to assist higher-education administrators, faculty, and researchers who wish to use the Research Experiences Instrument (REI). There are five aspects, or factors, that the REI is intended to measure relative to engineering Ph.D. students’ opportunities to practice being a professional in their research experiences, and an overall REI score. Detailed scoring instructions are provided. The REI was developed to assess how the research experiences of engineering Ph.D. students are preparing them for practice. Utilizing a rigorous instrument development process, the REI was shown to be a tool that can reliably and validly be used …

Assessing The Impact Of Ires On Researchers, Research Outcomes, And Students: A Case Study Approach, David B. Knight, Nicole Sanderlin, Kirsten A. Davis

School of Engineering Education Faculty Publications

IRES programs seek to develop globally connected future researchers while also facilitating broader long-term collaborations between U.S.-based and international research groups. Faculty and student exchanges are expected to result in international linkages between the researchers that bring new insights and methods to U.S.-based research projects, leading to transformative research. In this project, we conducted a comprehensive evaluation of IRES programs through a case study analysis. The nine selected cases focused specifically on engineering-focused IRES experiences for ease of comparison. Case studies included interviews with all U.S. PI’s, a selection of additional international and U.S. faculty researchers, and select student alumni …

Engineering Education Research, Michael C. Loui, Maura Borrego

School of Engineering Education Faculty Publications

This chapter describes several aspects of engineering education research with an emphasis on how they might relate to computing education research. We briefly summarize the history of engineering education as a scholarly field, and we describe the current structures that support engineering education research: academic departments, scholarly journals, annual conferences, and professional societies. We identify the theories that often inform engineering education research studies, including theories of cognition, motivation, and identity. We explain how quantitative, qualitative, and mixed methods have been used. We summarize the results of an illustrative selection of empirical studies across a broad range of topics, including …

An Assessment Framework For First-Year Introduction To Engineering Courses, Senay Purzer, Kerrie A. Douglas, Jill A. Folkerts, Taylor V. Williams

School of Engineering Education Faculty Publications

In this evidence-based practice paper, we describe an assessment framework that applies to first-year introductory engineering courses. First-year engineering courses cover a variety of learning objectives that address both technical and professional outcomes outlined in ABET. These courses also often involve open-ended design and modeling projects. The assessment of multiple competencies along with open-ended design can be a challenging task for educators. In this paper, we describe a framework that guides instructional processes for effective assessment for student learning. This assessment-centered teaching and learning framework helps connect specific learning objectives to broader learning goals or competencies and on-going formative feedback …

Board # 102 : Pecase: Implementing K-12 Engineering Standards Through Stem Integration - An Executive Summary Of The Products And Research, Tamara Moore, Kristina Tank, Aran Glancy, Elizabeth Gajdzik

School of Engineering Education Faculty Publications

This executive summary of the grant, PECASE: Implementing K-12 Engineering Standards through STEM Integration, comes at the conclusion of the project. The purpose of the grant was to develop a definition and explore the practice of engineering in K-12 STEM classrooms. The definition was then used to assess curricula, policy documents, teacher practice, and student learning. Through this work, the definition was then used to help with the framing and development of curricula for K-5 classrooms. The resulting curricula are called the PictureSTEM units. These instructional units for K-5 classrooms utilize engineering design and picture books to teach young students …

A Content Analysis Of How Engineering Is Assessed In Published Curricula, Kerrie A. Douglas, Tamara J. Moore, Hillary Merzdorf, Tingxuan Lee, Amanda C. Johnston

School of Engineering Education Faculty Publications

A Content Analysis of How Engineering is Assessed in Published Curricula (Fundamental)

The purpose of this proposal is to present research findings concerning how and what about engineering is commonly assessed in well-known engineering or integrated STEM published curriculum. Two of the major shifts brought about by Next Generation Science Standards (NGSS) are an increased emphasis in students’ capabilities to perform higher-level reasoning skills and integrate content understanding into science practices. At the same time, NGSS has made engineering integration into science education a priority, and it is an exciting time of reform as schools are exploring curriculum resources and …

A Cross-Sectional Study Of Engineering Identity During Undergraduate Education, Allison Godwin, Walter C. Lee

School of Engineering Education Faculty Publications

This research paper explores students’ engineering, mathematics, and physics identities across the four years of undergraduate engineering education. The focus of this work are subject-related role identities, or how students position themselves and are positioned by others as the kind of people that engage in engineering, mathematics, or physics. An identity as an engineer is a role identity because it embodies a specific character that an individual plays within a social sphere. In this case, the social sphere is in the process of becoming an engineer. Other research has focused on identity development of engineering students, often either early in …