Education Commons^™

Identifying Support Structures Associated With Informal Formative Evaluation In Instructional Design, Suzanne Smith, Tian Luo

STEMPS Faculty Publications

Instructional designers perform informal formative evaluation in design practice. An evaluation may be used to locate errors in alignment of instructional objectives or to increase the quality or effectiveness of a design. An instructional design review is similar to peer reviews in higher education which are often structured, and tools are provided to contribute to the review. A study was performed to identify the support structures and tools that contribute to building a community of feedback within the practice of instructional design reviews. Six instructional designers and design managers were interviewed to gather the processes they use in design reviews …

Report On Odu Findings From The Virginia Course Materials Survey Fall 2021, Karen Vaughan

Libraries Reports and Documents

From Introduction:

From October 1-December 31, 2021, the Virtual Library of Virginia (VIVA) conducted a survey of students at Virginia colleges and universities. More than 5,600 valid student responses from 41 institutions were received, reflecting an overall response rate of 10%. The largest response (58.5%) was from doctoral institutions, with private, two-year, and four-year institutions responding at 12-14%. A random sample of ODU students produced 287 responses (see Appendix A for ODU demographics). The survey aimed to answer these questions:

• What is the impact of course material costs on educational equity among Virginia students?
• What course content materials do students …

Caregivers' Perceptions Of Covid-19 Educational Disruptions On Children With Developmental Language Disorder And Typically Developing Peers, Katherine Radville, Danika Pfeiffer, Karynn Sheranian, Julie Wolter, Jessie Ricketts, Tiffany L. Hogan

Communication Disorders & Special Education Faculty Publications

Purpose: Understanding the experiences of families of children with developmental language disorder (DLD) during COVID-19 educational disruptions is essential for designing responsive supports during pandemic recovery efforts and beyond. This qualitative study describes the experiences of families of first-and second-grade children with DLD during the pandemic as compared to the experiences of families of typically developing (TD) peers.

Method: A conventional content analysis approach was used to analyze caregivers’ written responses to open-ended questions regarding their perceptions of COVID-19 educational disruptions. Responses were analyzed separately by group: caregivers of children with DLD (n= 23) and caregivers of TD children (n= …

Social Capital Development In Community College Alumni Who Studies Abroad, Heidi Fischer, Rosalind Latiner Raby

Educational Foundations & Leadership Faculty Publications

This qualitative study explores how US community college students who studied abroad use their existing social capital to enhance their engagement with other students and host country residents. The study also shows that students who studied abroad develop new social capitals for bridging and bonding with others. Data collection consisted of two rounds of in-depth interviews with 27 alumni students as well as a survey on participant background. Findings show that students use their social capital to enhance their relationships with other students and host country individuals. These positive relationships, in turn, enrich and build new social capitals that remained …

Authentic Learning Design Failures: The Need For Learner And Contextual Analysis And Participatory Design, Mohan Yang, Jon Harbor

STEMPS Faculty Publications

This design case examines what program leaders learned from failures in the design of a program of authentic learning about teaching diverse audiences through educational outreach. The program was initiated and then redesigned to develop the teaching and communication skills of graduate students from a wide range of backgrounds by engaging them in authentic experiences with middle school teachers and students. Analysis of post-program data revealed seven design failures related to the lack of upfront analysis to inform the program design. Each design failure was detailed through a fishbone diagram method to identify the corre- sponding contributing factors. The failures …

Predicting Language Performance From Narrative Language Samples, Kimberly A. Murphy, Alisha P. Springle, Mollee J. Sultani, Autumn Mcilraith, Language And Reading Research Consortium (Larrc)

Communication Disorders & Special Education Faculty Publications

Purpose: Analysis of narrative language samples is a recommended clinical practice in the assessment of children’s language skills, but we know little about how results from such analyses relate to overall oral language ability across the early school years. We examined the relations between language sample metrics from a short narrative retell, collected in kindergarten, and an oral language factor in grades kindergarten through 3. Our specific questions were to determine the extent to which metrics from narrative language sample analysis are concurrently related to language in kindergarten and predict language through Grade 3. Method: Participants were a sample …

Reconfigurable Modules For Automatic Creating Robotic Courses, Ana Djuric, Vukica Jovanovic

Engineering Technology Faculty Publications

The main objective of this paper is to describe a teaching tool that can be used for automated creation of different problems that can be used for assessing student knowledge while teaching industrial robotics courses. These materials can be used for instructors who need to create customized questions and answers for different student homework. Wayne State University, Detroit, MI offer different industrial robotics and courses related to fundamental robotic theory: kinematics, dynamics, and control. Old Dominion University, Norfolk, VA offers the course in Introduction to Industrial Robotics. Both programs are Mechanical Engineering Technology under the Engineering Technology departments / division. …

Questionable Research Practices Among Researchers In The Most Research-Productive Management Programs, Sven Kepes, Sheila K. Keener, Michael A. Mcdaniel, Nathan S. Hartman

Management Faculty Publications

Questionable research practices (QRPs) among researchers have been a source of concern in many fields of study. QRPs are often used to enhance the probability of achieving statistical significance which affects the likelihood of a paper being published. Using a sample of researchers from ten top research-productive management programs, we compared hypotheses tested in dissertations to those tested in journal articles derived from those dissertations to draw inferences concerning the extent of engagement in QRPs. Results indicated that QRPs related to changes in sample size and covariates were associated with unsupported dissertation hypotheses becoming supported in journal articles. Researchers also …

Determining The Difficulty And Discrimination Parameters Of A Mathematics Performance-Based Assessment, Abraham Gyamfi, Douglas G. Wren

Educational Foundations & Leadership Faculty Publications

Performance-based assessment (PBA) is different from traditional testing methods in that PBA presents real-life problems for students to solve by integrating critical thinking with their content knowledge and skills. Implementing PBA regularly in mathematics classes is associated with improved student achievement and motivation to learn; however, there are concerns about the general lack of psychometric data to support the use of performance assessments. To address such concerns, this study applied item response theory to estimate the difficulty and discrimination indices of items that comprised a newly developed mathematics PBA. Data were collected by administering the PBA to 750 senior high …

Leading Through Partnership: An Examination Of Longitudinal Trends In A School-University Partnership, Kala Burrell-Craft, Patrick Doyle, Mary Gregory, Jori S. Beck, Peter D. Wiens

Teaching & Learning Faculty Publications

The purpose of this manuscript is to detail pre- and post- changes that occurred after a principal, Dr. D, was moved from one middle school to another. Dr. D brought an established partnership with him to his new school including a teacher residency program, UTeach replication program, and an on-site course. The results of ANOVAs are presented regarding the school climate according to student and teacher perspectives. Significant results were found on a number of variables including student belonging, bullying, and teacher autonomy. This manuscript addresses the National Association for Professional Development Schools Essential #5: Research and Results.

Making A Difference Through Sustained In-Service Teacher Training, Abha Gupta, Guang Lea Lee

Teaching & Learning Faculty Publications

This study is based on collaboration between a school and a university on professional development training of 4th and 5th grade elementary school teachers in a southeastern state in the USA. The study was three-pronged and focused on teacher knowledge, pedagogy, and student achievement. We examined how the building of teacher capacity affected the performance of underachieving students in math and literacy. Underachieving students were targeted with specific strategies, projects, problems solving stories, self-reflection, and higher-level thinking questions. Student performance was measured for literacy achievement, with quantitative and qualitative measures used for data collection purposes. Students showed progress over previous …

Conducting A Formative Evaluation On A Course-Level Learning Analytics Implementation Through The Lens Of Self-Regulated Learning And Higher-Order Thinking, Pauline S. Muljana, Tian Luo, Greg Placencia

STEMPS Faculty Publications

Self-regulated learning (SRL) and higher-order thinking skills (HOTS) are associated with academic achievement, but fostering these skills is not easy. Scholars have suggested an alternative way to scaffold these important skills through learning analytics (LA). This paper presents a formative evaluation of a course-level LA implementation through the lens of self-regulated learning (SRL) and higher-order thinking skills (HOTS). We explored the changes in students’ SRL, HOTS, and perceptions at the end of the course term. Results indicate an increase in some elements of SRL and HOTS, and positive student perceptions. Discussion on implications and opportunities for informing future teaching strategies …

The Influence Of Participation In A Multi-Disciplinary Collaborative Service Learning Project On The Effectiveness Of Team Members In A 100-Level Mechanical Engineering Class, Stacie I. Ringleb, Pilar Pazos, Julia Noginova, Francisco Cima, Orlando Ayala, Krishnanand Kaipa, Jennifer Kidd, Kristie Gutierrez

Mechanical & Aerospace Engineering Faculty Publications

Engineers need to develop professional skills, including the ability to work successfully in teams and to communicate within and outside of their discipline, in addition to required technical skills. A collaborative multi-disciplinary service learning project referred to as Ed+gineering was implemented in a 100-level mechanical engineering course. In this collaboration, mechanical engineering students, primarily in the second semester of their freshman year or first semester of their second year, worked over the course of a semester with education students taking a foundations course to develop and deliver engineering lessons to fourth or fifth graders. Students in comparison engineering classes worked …

Review Of: Estrellado, R. A., Freer, E. A., Motsipak, J., Rosenberg, J. M., & Velásquez, I. C. (2021). Data Science In Education Using R. London, England: Routledge, Esther Michela, Robert L. Moore

STEMPS Faculty Publications

Data use in education is continually expanding in scope (Romero and Ventura 2020), and though programming resources are plentiful, those that truly scaffold the learning process are scarce. Data Science in Education Using R by Estrellado, Freer, Motsipak, Rosenberg, and Velásquez is a combination primer and workbook. This format allows the volume to serve as both an introductory R resource aimed to help readers overcome the daunting initial roadblocks to proficiency and familiarize them with educational data uses. We reviewed this book from the perspective of a beginning R user with little other programming background for whom this book is …

Strengthening The Connections Between Library Instruction And Student Success, Lucinda R. Wittkower, Joleen Westerdale Mcinnis

Libraries Faculty & Staff Presentations

This poster will share the results from a three-year study that investigated correlations between student participation in library instruction and student academic achievement. Using rigorous research practice and acknowledging ethical concerns by use of an in-depth student consent process, the researchers will share their findings regarding the relationships that exist between student participation in library instruction and completion of course for which instruction was attended and the relationships between student participation in library instruction and their grade in course. Additionally, the researchers will share the unexpected, but interesting finding related to student withdrawal rates. Finally, the researchers will include information …

Using Self-Based Methodologies To Unpack Mathematics Teacher Educators' Work, Elizabeth Suazo-Flores, Signe E. Kastberg, Melva R. Grant, Jennifer Ward, Sue Ellen Richardson, Olive Chapman

Teaching & Learning Faculty Publications

Narrative inquiry, self-study, and autoethnography (i.e., self-based methodologies) are methodologies used by mathematics teacher educators (MTEs). These methodologies have opened up the field by unpacking and unearthing MTEs' work communicating findings from their practices. Building from our previous working groups at PME-NA 2018-2020, we sustain a community where MTEs can feel supported in their study design, implementation, representation of findings, and publication using self-based methodologies. At PME-NA Philadelphia, we will continue our work at PME-NA Mexico on self-based methodologies to develop perspectives on philosophical underpinnings of self-based methodologies and addressing trustworthiness and authenticity in our reports.

Adding "Student Voice" To The Mix: Perception Surveys And State Accountability Systems, Jack Schneider, James Noonan, Rachel S. White, Douglas Gagnon, Ashley Carey

Educational Foundations & Leadership Faculty Publications

For the past two decades, student perception surveys have become standard tools in data collection efforts. At the state level, however, “student voice” is still used sparingly. In this study, we examine the ways in which including student survey results might alter state accountability determinations. Reconstructing the accountability system in Massachusetts, we draw on a unique set of student survey data, which we add to the state’s formula at a maximally feasible dosage in order to determine new school ratings. As we find, student survey data shift school accountability ratings in small but meaningful ways and appear to enhance functional …

Understanding And Predicting Retractions Of Published Work, Sai Ajay Modukuri, Sarah Rajtmajer, Anna Cinzia Squicciarini, Jian Wu, C. Lee Giles

Computer Science Faculty Publications

Recent increases in the number of retractions of published papers reflect heightened attention and increased scrutiny in the scientific process motivated, in part, by the replication crisis. These trends motivate computational tools for understanding and assessment of the scholarly record. Here, we sketch the landscape of retracted papers in the Retraction Watch database, a collection of 19k records of published scholarly articles that have been retracted for various reasons (e.g., plagiarism, data error). Using metadata as well as features derived from full-text for a subset of retracted papers in the social and behavioral sciences, we develop a random forest classifier …

Assessing An Academic Readiness Model In A Community College Occupational Program, Scott A. Hauert, Robert L. Moore, Quinton Nottingham

STEMPS Faculty Publications

This study assessed a localized academic readiness model that measures academic skills relevant to an occupational program and the institutional context. Enrollment in the program’s mandatory gateway course is contingent upon establishing academic readiness either: (1) possess an associate-level or higher degree; (2) achieve placement test scores that permit enrollment in college-level English composition/critical reading; or (3) complete developmental education prerequisites to permit enrollment in college-level English composition/critical reading. Using the gateway course performance for 112 students, the study explored whether any of the model’s criterions are significantly related to success in the gateway course and whether there is any …

Informing Makerspace Outcomes Through A Linguistic Analysis Of Written And Video-Recorded Project Assessments, Kevin M. Oliver, Jennifer K. Houchins, Robert L. Moore, Chuang Wang

STEMPS Faculty Publications

A growing body of research focuses on what outcomes to assess in makerspaces, and appropriate formats for capturing those outcomes (e.g. reflections, surveys, and port-folios). Linguistic analysis as a data mining technique holds promise for revealing different dimensions of learning exhibited by students in makerspaces. In this study, student reflections on makerspace projects were gathered in 2 formats over 2 years: private written assessments captured in the 3D GameLab gamification platform, and semi-public video-recorded assessments posted in the more social FlipGrid platform. Transcripts of student assessments were analyzed using Linguistic Inquiry Word Count (LIWC) to generate 4 summary variables thought …

Mathematics Teacher Educators Using Self-Based Methodologies, Elizabeth Suazo-Flores, Jennifer Ward, Sue Ellen Richardson, Melva R. Grant, Dana Cox, Signe E. Kastberg, Olive Chapman

Teaching & Learning Faculty Publications

Narrative inquiry, self-study, and autoethnography (i.e., self-based methodologies) are becoming a more common choice of mathematics teacher educators (MTEs). This has opened new possibilities and challenges for early career MTEs as they try to disseminate their findings in mathematics education journals. Building from our working group at PME-NA 2018 and 2019, we respond to the need for creating a community where MTEs can feel supported in their study design, implementation, representation of findings, and publication using self-based methodologies. This year, we continue our focus on mentoring and scholarship on self-based methodologies. We invite English- and Spanish-speaking MTEs with research projects …

A Continuum Of Data Literacy For Teaching, Jori S. Beck, Diana Nunnaley

Teaching & Learning Faculty Publications

Accountability for education in the United States has existed since the first formalized system of education. Although this accountability is an important part of society, these systems must be expanded beyond high stakes assessments to include other types of data including P-12 student voices. The purpose of the current manuscript is to present a continuum for data literacy for teachers that spans preservice to inservice teacher education. We conceptualize data literacy for teachers as a metaconstruct that includes the construct of assessment literacy. The research on enabling and marginalizing factors and exposure to data are reviewed at the preservice and …

A Continuum Of Critical Consciousness: Exploring One Resident's Concerns, Jori S. Beck

Teaching & Learning Faculty Publications

In the United States, there is a demand for richer clinical teacher education experiences. Partially in response to this call, innovative new programs like teacher residencies are being developed. As teacher preparation programs are shaped by these mandates, researchers must respond to shifts in the field. The current manuscript includes data from a resident, or teacher candidate, enrolled in a residency program—specifically, his yearlong apprenticeship. Using interviews and other qualitative data, the author examined how the resident’s concerns shifted and also how these concerns differed from those uncovered in previous research. Specifically, his concerns appeared to be more dynamic than …

“There Is Subjectivity, There Is Bias”: Teacher Candidates’ Perceptions Of Equity In Data Literacy For Teaching, Heather Whitesides, Jori S. Beck

Teaching & Learning Faculty Publications

Research on equity in data literacy for teaching has lagged yet is of critical importance to ensuring new teachers are prepared to serve diverse students. Our multiple case study conveyed four elementary teacher candidates’ understandings of this construct and their reaction to instruction in this domain. Data collection included interviews, item analysis, and concept maps. Our participants developed a broader view of data by the end of the course, but often did not recognize inequitable data practices like tracking which conveys a misalignment between beliefs and practices. We explored implications for policy and practice based on our findings.

Assessing Teacher Pedagogical Knowledge: The Video Assessment Of Teacher Knowledge (Vatk), Peter D. Wiens, Jori S. Beck, Christina J. Lunsmann

Teaching & Learning Faculty Publications

This study examines the creation and implementation of a video-based assessment of teacher knowledge of effective teaching called the Video Assessment of Teacher Knowledge (VATK). The VATK was developed with the intention of creating an easily administered, standardised assessment that can potentially provide information on teacher candidate knowledge of teaching at important points during their training programmes and into the teaching field. The theoretical framework upon which the VATK was developed and the process for item creation are described. Classical test theory and item response theory analysis were conducted to determine the measure’s potential for use in future studies. Analyses …

Experiences Of African American Teachers In Desegregated Pk–12 Schools: A Systematic Literature Review, Yonghee Suh, Brian J. Daugherity, Jihea Maddamsetti, Angela Branyon

Teaching & Learning Faculty Publications

This literature review reports findings from 19 empirical studies on the experiences of African American teachers in PK–12 desegregated schools. The research questions were: What do we know about the experiences of African American teachers in desegregated PK–12 schools? What are the challenges African American teachers experience in desegregated PK–12 schools? In response to these questions, the article first discusses school desegregation after the Brown v. Board of Education decision and its impact on African American teachers as a historical backdrop. Findings from 19 studies were analyzed through grounded theory. Two core themes were identified from our findings: persistent structural …

The Power Of Open: Benefits, Barriers, And Strategies For Integration Of Open Educational Resources, Tian Luo, Kirsten Hostetler, Candice Freeman, Jill Stefaniak

STEMPS Faculty Publications

Open Educational Resources (OER) are becoming a significant, mission-driven trend within educational literature. To help address rising costs, instructors and designers are looking to OER to effectively replace traditional instructional content, which requires more than just identify and replace. Drawing from 51 OER studies conducted in countries across the five continents, this systematic literature review explored the empirical themes evident in the current research on a global scale. This review found (1) discoverability, sustainability, and remixing are significant barriers that stand in the way of OER disrupting traditional textbook models; (2) there is no significant difference in learning outcomes when …

Improving Classroom Management And Teacher Retention: A Needs Assessment, Jill Stefaniak, Jilian L. Reynolds, Tian Luo

STEMPS Faculty Publications

This case explores how a needs assessment was conducted at a middle school experiencing high rates of teacher turnover. Pamela Frost, an experienced instructional designer, was assigned to assess the situation and identify opportunities to improve professional development opportunities for the teachers. As a part of a needs assessment, Pamela gathered data to address needs pertaining to classroom management challenges, teacher attrition rates, and establishing relations with the local community. This case explores how Pamela gathered data and triangulated her findings to determine what interventions were needed.

School Librarians Making Global Connections: Conjecture Mapping And Researcher Practitioner Partnerships, Barbara Schultz-Jones, Marcia Mardis, Sue Kimmel

STEMPS Faculty Publications

Dr. Barbara Schultz-Jones (University of North Texas), Dr Marcia Mardis (Florida State University) and Dr Sue Kimmel (Old Dominion University, Virginia) share their research on a way to support evidence building of causal relationships between the school library and a teacher librarian’s contribution to student outcomes.

Well Begun Is Half Done: Using Online Orientation To Foster Online Students' Academic Self-Efficacy, M'Hammed Abdous

Distance Learning Faculty & Staff Publications

Past research suggests that the use of an online learning orientation is an effective proactive strategy to ease online students' transition into online learning. Based on a sample of 3,888 online students from an urban public university, we used ordinal logistic regression to understand the influence of students' satisfaction with an online learning orientation (OLO), their prior level of online learning experience, and their demographics on their academic self-efficacy (ASE). Consistent with prior research, our findings confirmed the influence of students' satisfaction with OLO, their prior online learning experience, and their gender on their ASE. Unsatisfied students were 85% less …