Education Commons^™

Assessing The Quality Of Doctoral Dissertation Literature Reviews In Instructional Technology, Melynda Harrison Fitt, Andrew Walker, Heather Leary

Instructional Technology and Learning Sciences Faculty Publications

Assessment of the doctoral dissertation literature review provides insight into a student’s preparation for future work as a researcher. In 2004, efforts to assess the quality of literature reviews in doctoral dissertations were pioneered by Boote & Beile. Their work represents an important response to the call for improved research skills among emerging scholars. The purpose of this study is to replicate their work in a focused area of educational research, specifically Instructional Technology, and to examine the inter-rater reliability of the rubric. The findings suggest that dissertation literature reviews in Instructional Technology show the same need for improvement as …

The Role Of Learner Attributes And Affect Determining The Impact Of Agent Presence, Yanghee Kim

Yanghee Kim

This paper introduces two experimental studies that examined the potential of animated virtual peers (VP) to build social relations with learners in online learning environments. VP emotions and learner characteristics were foci of interest. Study I investigated the impact of VP emotional expressions (positive vs. negative vs. neutral) and gender and learner gender on college students’ perceptions of agent persona, motivation, and learning. Study II investigated the interaction effects of VP presence and learner gender and learner sociability on high-school students’ task-related attitudes, self-efficacy beliefs, and learning. Overall, the results revealed the interaction effects of VP/learner attributes on the learners’ …

A Problem Based Learning Meta Analysis: Differences Across Problem Types, Implementation Types, Disciplines, And Assessment Levels, Andrew Walker, Heather Leary

Instructional Technology and Learning Sciences Faculty Publications

Problem based learning (PBL) in its most current form originated in Medical Education but has since been used in a variety of disciplines (Savery & Duffy, 1995) at a variety of educational levels (Savery, 2006). Although recent meta analyses have been conducted (Dochy, Segers, Van den Bossche, & Gijbels, 2003; Gijbels, Dochy, Van den Bossche, & Segers, 2005) that attempted to go beyond medical education, they found only one study in economics and were unable to explain large portions of the variance across results. This work builds upon their efforts as a meta-analysis that crosses disciplines as well as categorizes …

Developing A Review Rubric For Learning Resources In Digital Libraries, Heather Leary, Sarah Giersch, Andrew Walker, Mimi Recker

Instructional Technology and Learning Sciences Faculty Publications

This paper describes the development of a review rubric for learning resources in the context of the Instructional Architect (IA), a web-based authoring tool. We describe the motivation for developing a review rubric, the process for creating it by synthesizing the rubrics of other education-related digital libraries, and the results of testing the rubric with teachers. Analysis of usability and reliability indicates that the review rubric influences how teachers design online learning resources.

Validity And Problem-Based Learning Research: A Review Of Instruments Used To Assess Intended Learning Outcomes, Brian Robert Belland, Brian F. French, Peggy A. Ertmer

Instructional Technology and Learning Sciences Faculty Publications

Problem-based learning (PBL) spread from the medical school to other university and K-12 contexts due, in part, to the stated promise that PBL produces the target outcomes of deep content learning, increased problem-solving ability, and increased self-directed learning (Hmelo-Silver, 2004). However, research results have been unclear. This paper examines how the three target outcomes of PBL were measured in 33 empirical studies. Results indicate that few studies included 1) theoretical frameworks for the assessed variables and constructs, 2) rationales for how chosen assessments matched the constructs measured, or 3) other information required for readers to assess the validity of authors’ …

A Connective Ethnography Of Peer Knowledge Sharing And Diffusion In A Tween Virtual World, Deborah A. Fields, Y. B. Kafai

Instructional Technology and Learning Sciences Faculty Publications

Prior studies have shown how knowledge diffusion occurs in classrooms and structured small groups around assigned tasks yet have not begun to account for widespread knowledge sharing in more native, unstructured group settings found in online games and virtual worlds. In this paper, we describe and analyze how an insider gaming practice spread across a group of tween players ages 9–12 years in an after-school gaming club that simultaneously participated in a virtual world called Whyville.net. In order to understand how this practice proliferated, we followed the club members as they interacted with each other and members of the virtual …

What Do Students Gain From A Week At Science Camp? Youth Perceptions And The Design Of An Immersive Research-Oriented Astronomy Camp, Deborah A. Fields

Instructional Technology and Learning Sciences Faculty Publications

This study explored American high school students’ perceptions of the benefits of a summer astronomy camp, emphasizing a full cycle of the research process and how the organization of the camp contributed to those perceptions. Semi-structured interviews with students and staff were used to elicit the specific benefits that campers perceived from their experiences and examine them in relation to the stated goals and strategies of camp staff. Among the perceived benefits that students described were peer relationships, personal autonomy, positive relationships with staff, and deepened science knowledge. These perceived benefits appear to influence the kinds of identities students constructed …

Effective Practices Of Project Lead The Way Partnership Teams, Cody J. Reutzel

All Graduate Theses and Dissertations, Spring 1920 to Summer 2023

The purpose of this study was to gather information from Project Lead The Way (PLTW) partnership team experts. This project follows the methodology of a modified Delphi study. A review of literature in the areas of curriculum development, pre-college engineering, and the Delphi research technique provided the background for the structure utilized. Top programs from across the country were questioned to identify and come to a consensus on top components essential to developing and utilizing a successful PLTW partnership team. The components were categorized into two lists: effective practices utilized to make a program successful and effective practices employed by …

Problem-Based Educational Games: Connections, Prescriptions, And Assessment, Andrew Walker, Brett E. Shelton

Instructional Technology and Learning Sciences Faculty Publications

The overwhelming success of the commercial game market has brought increased attention to emerging work in educational game design. Much of the existing work in educational games a strong similarity to the field of Problem-Based Learning (PBL), which has a rich history of conceptual literature as well as empirical investigations. Despite apparent similarities between the two fields, there has been no formal effort to explore the connections between them. This conceptual paper examines the basic tenants of PBL with an eye toward making prescriptive recommendations for the design and use of problem-based educational games. Examples within existing educational games are …

Creating Pedagogical Agents As Social Models In An Online Learning Environment Mathgirls, Yanghee Kim, B. Xu, A. Sharif

Yanghee Kim

This paper introduces the learning environment MathGirls for high school girls learning fundamentals of algebra. Grounded in social cognitive theories of learning, MathGirls utilizes pedagogical agents to create a girl-friendly virtual learning environment. The design constituents of pedagogical agents are reviewed. These constituents are likely to influence building agent/learner relations. The agent design and system architecture of the MathGirls environment are developed to integrate some of the design constituents. Empirical findings from MathGirls deployment in classrooms support the efficacy of the presence of pedagogical agents in shaping affective and cognitive characteristics of the learner. The paper concludes with the discussions …

Unleashing The Usefulness Of Educational Resources Through Mining Of Educational Metadata, Anne R. Diekama, Jennifer A. Bailey, Blythe A. Bennett, Holly Devaul

Instructional Technology and Learning Sciences Faculty Publications

While there is a strong movement to develop new educational resources to bring students to the competencies represented by educational content standards, it is recognized that there are vast repositories of educational resources already developed that are suitable to address those competencies. However, these resources need to be indexed by national and state standards to make them accessible for teachers who are increasingly required to teach to certain educational standards (Diekema and Chen, 2005). In the early 1980s, a perceived a crisis in the American education system encouraged the creation of national standards by professional subject-area organizations such as the …

The Design And Use Of Simulation Computer Games In Education, Brett E. Shelton, David A. Wiley

Instructional Technology and Learning Sciences Faculty Publications

This book, edited by Brett Shelton and David Wiley, is a view of models and simulations for education and research.

Table of Contents

1. In Praise of Epistemology - David Shaffer

2. Six Ideas in Search of a Discipline - Richard Van Eck

3. Building Bridges Between Serious Game Design and Instructional Design - Jamie Kirkley, Sonny Kirkley and Jerry Heneghan

4. Layered Design in an Instructional Simulation - Andrew S. Gibbons and Stefan Sommer

5. Designing Educational Games for Activity-Goal Alignment - Brett E. Shelton

6. "The Peripatos Could Not have Looked Like That," and Other Educational Outcomes From …

The Underutilization Of Internet And Communication Technology-Assisted Collaborative Project-Based Learning Among International Educators: A Delphi Study, Barry S. Kramer, Andrew Walker, Jennifer M. Brill

Instructional Technology and Learning Sciences Faculty Publications

This study explores the barriers associated with teachers implementing Internet and Communication Technology-assisted Collaborative Project-based Learning (ICTCPrjBL) as a classroom teaching methodology with students. We used a Web-based Delphi method to engage experienced educators in anonymous consensus building consisting of three rounds of surveys. The Round 1 analysis yielded 51 barriers. The Round 2 analysis produced descriptive statistics (range, mean, and standard deviation) on the importance of each barrier. The Round 3 analysis confirmed 16 of the 51 (31.4%) barriers as “moderately significant” to “very significant” to implementing ICTCPrjBL. Important contributions of this study include: (a) identification of barriers to …

Pedagogical Agents As Learning Companions: The Impact Of Agent Emotion And Gender, Yanghee Kim, A. L. Baylor, E. Shen

Yanghee Kim

The potential of emotional interaction between human and computer has recently interested researchers in human–computer interaction. The instructional impact of this interaction in learning environments has not been established, however. This study examined the impact of emotion and gender of a pedagogical agent as a learning companion (PAL) on social judgements, interest, self-efficacy, and learning. Two experiments investigated separately the effects of a PAL's emotional expression and empathetic response. Experiment 1 focused on emotional expression (positive vs. negative vs. neutral) and gender (male vs. female) with a sample of 142 male and female college students in a computer literacy course. …

Mathgirls: Toward Developing Girls’ Positive Attitude And Self-Efficacy Through Pedagogical Agents, Yanghee Kim, Q Wei, B Xu, Y Ko, V Ilieva

Yanghee Kim

MathGirls is a pedagogical-agent-based environment designed for high-school girls learning introductory algebra. Since females are in general more interested in interactive computing and more positive about the social presence of pedagogical agents, the environment provides a girl-friendly social learning environment, where pedagogical agents encourage the girls to build constructive views of learning math. This study investigated the impact of agent presence on changes in the girls’ math attitude, their math self-efficacy, and their learning; on the girls’ choice of their agents; and, on their perceptions of agent affability. The results revealed that the girls with an agent developed a more …

Pedagogical Agents As Social Models To Influence Learner Attitudes, Yanghee Kim, A. Baylor

Instructional Technology and Learning Sciences Faculty Publications

Based on social-cognitive theory, we describe the role of pedagogical agents as “social models.” In several experimental studies we have found that pedagogical agents as social models can effectively persuade and motivate learners. We briefly describe two on-going projects where agents as social models are employed to enhance young women’s motivation and attitudes toward math and engineering.

Desirable Characteristics Of Learning Companions, Yanghee Kim

Instructional Technology and Learning Sciences Faculty Publications

This study investigated the desirable characteristics of anthropomorphized learning-companion agents for college students. First, interviews with six undergraduates explored their concepts of desirable learning companions. The interviews yielded agent competency, agent personality, and interaction control. Next, a controlled experiment examined whether learner competency (strong vs. weak) would relate directly to agent competency (high vs. low) and to interaction control (agent-control vs. learner-control). The dependent measures included learners' perceptions of agent functionality, their self-efficacy beliefs in the task, and their learning. The results indicated that academically strong students perceived the high-competent agent higher than the lowcompetent agent and showed higher self-efficacy …

Text Categorization For Aligning Educational Standards, O. Yilmazel, N. Balasubramanian, S. C. Harwell, J. Bailey, Anne R. Diekama, E. D. Liddy

Instructional Technology and Learning Sciences Faculty Publications

Standard alignment (where standards describing similar concepts are correlated) is a necessary task in providing full access to educational resources. Manual alignment is time consuming and expensive. We propose an automatic alignment system, using machine learning techniques utilizing natural language processing. In this paper we discuss our experiments on text categorization for automatic alignment. We explore the role of relevant vocabulary sets in automatic alignment.

Designing Educational Games For Activity-Goal Alignment, Brett E. Shelton

Instructional Technology and Learning Sciences Faculty Publications

What indeed, can we expect from our newest trend in education, implementing moving pictures and conversations with instruction through simulation games? Lewis Carroll's familiar narratives Alice's Adventures in Wonderland and Alice Through the Looking Glass provide helpful imagery for many of the queries, explorations and assumptions we currently make about this latest Wonderland of academia. So what are the goals for the designers and researchers of educational games, or perhaps more importantly, what should be the goals?

The Competencies And Characteristics Required Of An Effective Project Manager: A Web-Based Delphi Study, Jennifer M. Brill, M. J. Bishop, Andrew Walker

Instructional Technology and Learning Sciences Faculty Publications

This study explores the competencies required for a project manager to be effective in the workplace. We used a Web-based Delphi method to lead experienced project managers through an anonymous consensus-building process consisting of two rounds of surveys. The Round I analysis of 147 respondents, all with 20 or more years of project management experience, yielded 117 project management success factors, 78 of which were identified as “trainable” competencies. The Round II analysis confirmed 42 of the 78 competencies (53.8%) as “very important” to “extremely important” to project manager success. Important contributions of this study include: (a) reporting on project …

A Social-Cognitive Framework For Designing Pedagogical Agents As Learning Companions, Yanghee Kim, Amy L. Baylor

Yanghee Kim

Teaching and learning are highly social activities. Seminal psychologists such as Vygotsky, Piaget, and Bandura have theorized that social interaction is a key mechanism in the process of learning and development. In particular, the benefits of peer interaction for learning and motivation in classrooms have been broadly demonstrated through empirical studies. Hence, it would be valuable if computer-based environments could support a mechanism for a peer-interaction. Though no claim of peer equivalence is made, pedagogical agents as learning companions (PALs) -- animated digital characters functioning to simulate human-peer-like interaction -- might provide an opportunity to simulate such social interaction in …

Content-Based English Learning Through Pedagogical Agents, Yanghee Kim, P. Punahm, Y. Ko

Yanghee Kim

This paper suggests how an advanced technology called pedagogical agents can be applied to English education to benefit learners across ages through computer-assisted language learning (CALL) and content-based language learning (CBLL). CALL, when designed appropriately, has positively influenced the development of a learner’s linguistic proficiency and communicative competence (Chun, 1994; Fotos & Browne, 2004). CBLL integrates language learning with subject-matter learning to make language learning more meaningful (Snow, 2001; Swain, 1998). However, the conventional CALL programs are often criticized for the lacking a social context, considered essential for successful language learning ( Warschauer, 2004). Also, CBLL seems rarely applied to …

Pedagogical Agents As Learning Companions: The Role Of Agent Competency And Type Of Interaction, Yanghee Kim, Amy L. Baylor, Pals Group

Yanghee Kim

This study was designed to examine the effects of the competency (low vs. high) and interaction type (proactive vs. responsive) of pedagogical agents as learning companions (PALs) on learning, self-efficacy, and attitudes. Participants were 72 undergraduates in an introductory computer-literacy course who were randomly assigned to one of four treatments: Low-Proactive, Low-Responsive, High-Proactive, and High-Responsive. Results indicated a main effect for PAL competency. Students who worked with the high-competency PAL in both proactive and responsive conditions achieved higher scores in applying what they had learned and showed more positive attitudes toward the PAL. However, students who worked with the low-competency …

A Design-Based Research Strategy To Promote Scalability For Educational Innovations, Jody Clarke-Midura, C. Dede, D. J. Ketelhut, B. Nelson, C. Bowman

Instructional Technology and Learning Sciences Faculty Publications

This article offers insights into how the design of innovations can enhance their “scalability”: the ability to adapt an innovation to effective usage in a wide variety of contexts, including settings where major conditions for success are absent or attenuated. We are implementing the River City MUVE curriculum, a technology-based innovation designed to enhance engagement and learning in middle school science, in a range of educational contexts. Based on our studies of these scaling up activities, we offer examples of design strategies for scalability and describe our plan to develop a “scalability index.”

Perceptions Of The Value Of Problem-Based Learning Among Students With Special Needs And Their Teachers, Brian Robert Belland, P. A. Ertmer, K. D. Simons

Instructional Technology and Learning Sciences Faculty Publications

While problem-based learning (PBL) has been found to be effective with gifted and average students (Hmelo-Silver, 2004), little is known about its impact on students with special needs. This study examines the perceptions of middle-school students with mild, moderate, and severe disabilities and of their teachers regarding the value of participating in a PBL unit. The unit focused on the physical accessibility of a low-SES, rural community where the students’ school was located.We used the constant comparative method (Glaser & Strauss, 1967) to analyze interview data, and used observation data and artifacts to triangulate interview comments. Among the noteworthy findings …

Methodological Challenges For Identifying And Coding Diverse Knowledge Elements In Interview Data, Victor R. Lee, Moshe Krakowski, Bruce Sherin, Megan Bang, Gregory Dam

Instructional Technology and Learning Sciences Faculty Publications

This paper, as part of a symposium on the analysis of clinical interview data and the development of a framework for analyzing students' intuitive science knowledge, identifies and discusses methodological challenges encountered when specifying the knowledge elements and resources are invoked dynamically during a clinical interview. Drawing from interviews with middle school students about the seasons and an analysis of knowledge in terms of 'nodes', two classes of problems are identified: those associated with identification of nodes and those associated with their application as codes to a transcript-based data corpus. We posit that these challenges are common ones associated with …

Pedagogical Agents As Learning Companions: Building Social Relations With Learners, Yanghee Kim

Yanghee Kim

This study examined the potential of pedagogical agents as learning companions (PALs) to build social relations with learners and, consequently, to motivate learning. The study investigated the impact of PAL affect (positive vs. negative vs. neutral), PAL gender (male vs. female), and learner gender (male vs. female) on learners’ social judgments, motivation, and learning in a controlled experiment. Participants were 142 college students in a computer-literacy course. Overall, the results indicated the interaction effects of PAL affect, PAL gender, and learner gender on learners’ social judgments (p < .001). PAL affect impacted learners’ social judgments (p < .001) and motivation (p < .05). PAL gender influenced motivation (p < .01) and recall of learning (p < .05). Learner gender influenced recall of learning (p < .01). The implications of the findings are discussed.

Simulating Instructional Roles Through Pedagogical Agents, Amy L. Baylor, Yanghee Kim

Instructional Technology and Learning Sciences Faculty Publications

This paper describes the design and empirical validation of three distinct pedagogical agent roles (Expert, Motivator, and Mentor) for college students within the MIMIC (Multiple Intelligent Mentors Instructing Collaboratively) agent-based research environment. The pedagogical agent roles were operationalized by image, animation, affect, voice and script, and were developed in Poser 4 and implemented via Microsoft Agent. Two controlled experiments validated the instantiation of the three roles according to learner perception (N=78) and actual impact on motivation and learning (N=71). The results confirmed that the agent roles were not only perceived by the students to reflect their intended purposes but also …

Design-Based Research Strategies For Developing A Scientific Inquiry Curriculum In A Multi-User Virtual Environment, B. Nelson, D. J. Ketelhut, Jody Clarke-Midura, C. Bowman, C. Dede

Instructional Technology and Learning Sciences Faculty Publications

This National Science Foundation funded project is studying graphical multi-user virtual environments (MUVEs) to investigate whether using this interactive medium in classroom settings can simulate real-world experimentation and can provide students with engaging, meaningful learning experiences that enhance scientific literacy. In the project's River City curriculum, teams of middle school students are asked to collaboratively solve a digital 19th century city's problems with illness, through interaction with digital artifacts, tacit clues, and computer-based 'agents' acting as mentors and colleagues in a virtual community of practice. This article describes the design-based research strategy by which we are currently extending an educational …

Pedagogical Agents’ Personas: Which Affects More, Image Or Voice?, Yanghee Kim, A. L. Baylor, G. Reed

Yanghee Kim

The purpose of this study was to examine the effects of image and voice of pedagogical agents on student perception and learning. Pedagogical agents were developed with differing image (expert-like vs. mentor-like) and voice (strong vs. calm vs. computer-generated), but with identical gesture, affect, comments, and gender. 109 undergraduates in a computer literacy course were randomly assigned to one of the six conditions. The results revealed a significant main effect for agent image on role perception: the mentor-like image was perceived as more motivating, as hypothesized. Also, there was a significant main effect for voice: the strong voice was overall …