Engineering Commons^™

Advancing Omnimodality: Expanding Human Creativity Through Adaptable And Accessible Multimodal Computing Systems, Joshua Urban Davis

Dartmouth College Ph.D Dissertations

Emerging technologies have given us a whole host of new ways for people to be

creative. From the immersive worlds of AR/VR to the synthesis powers of largelanguage

models and generative AI, these new tools hold the potential to reshape

human expression and creativity. But how can we ensure that these new modalities

are accessible to everyone, even those who aren’t able bodied? This thesis advocates

for a human-centered approach to the development of many-modal systems. I will

probe how our machines support, direct, or inhibit creativity as a mode of problem

solving through 6 novel multimodal prototype interfaces and …

Diegetic Sonification For Low Vision Gamers, Jhané Dawes

Master's Theses

There are not many games designed for all players that provide accommodations for low vision users. This means that low vision users may not get to engage with the gaming community in the same way as their sighted peers. In this thesis, I explore how diegetic sonification can be used as a tool to support these low vision gamers in the typical gaming environment. I asked low vision players to engage with a prototype game level with two diegetic sonification techniques applied, without the use of their corrective lenses. I found that participants had more enjoyment and experienced less difficulty …

Evaluating Human Eye Features For Objective Measure Of Working Memory Capacity, Yasasi Abeysinghe, Enkelejda Kasneci (Ed.), Frederick Shic (Ed.), Mohamed Khamis (Ed.)

Computer Science Faculty Publications

Eye tracking measures can provide means to understand the underlying development of human working memory. In this study, we propose to develop machine learning algorithms to find an objective relationship between human eye movements via oculomotor plant and their working memory capacity, which determines subjective cognitive load. Here we evaluate oculomotor plant features extracted from saccadic eye movements, traditional positional gaze metrics, and advanced eye metrics such as ambient/focal coefficient , gaze transition entropy, low/high index of pupillary activity (LHIPA), and real-time index of pupillary activity (RIPA). This paper outlines the proposed approach of evaluating eye movements for obtaining an …

Who Is Missing? Characterizing The Participation Of Different Demographic Groups In A Korean Nationwide Daily Conversation Corpus, Haewoon Kwak, Jisun An, Kunwoo Park

Research Collection School Of Computing and Information Systems

A conversation corpus is essential to build interactive AI applications. However, the demographic information of the participants in such corpora is largely underexplored mainly due to the lack of individual data in many corpora. In this work, we analyze a Korean nationwide daily conversation corpus constructed by the National Institute of Korean Language (NIKL) to characterize the participation of different demographic (age and sex) groups in the corpus.

Computer Enabled Interventions To Communication And Behavioral Problems In Collaborative Work Environments, Ashutosh Shivakumar

Browse all Theses and Dissertations

Task success in co-located and distributed collaborative work settings is characterized by clear and efficient communication between participating members. Communication issues like 1) Unwanted interruptions and 2) Delayed feedback in collaborative work based distributed scenarios have the potential to impede task coordination and significantly decrease the probability of accomplishing task objective. Research shows that 1) Interrupting tasks at random moments can cause users to take up to 30% longer to resume tasks, commit up to twice the errors, and experience up to twice the negative effect than when interrupted at boundaries 2) Skill retention in collaborative learning tasks improves with …

Automatically Classifying Familiar Web Users From Eye-Tracking Data:A Machine Learning Approach, Meli̇h Öder, Şükrü Eraslan, Yeli̇z Yesi̇lada

Turkish Journal of Electrical Engineering and Computer Sciences

Eye-tracking studies typically collect enormous amount of data encoding rich information about user behaviours and characteristics on the web. Eye-tracking data has been proved to be useful for usability and accessibility testing and for developing adaptive systems. The main objective of our work is to mine eye-tracking data with machine learning algorithms to automatically detect users' characteristics. In this paper, we focus on exploring different machine learning algorithms to automatically classify whether users are familiar or not with a web page. We present our work with an eye-tracking data of 81 participants on six web pages. Our results show that …

Models And Methods Of Designing Human-Machine Interaction-Oriented Interfaces, Ozod Radjabov, Shakhrillo Bobokulov, Behruz Boboqulov, Hojiyev Sunatullo

Bulletin of TUIT: Management and Communication Technologies

Formalization approaches of user interface design (UID) in conjunction with model driven techniques aim to improve the usability in terms of conformity to standards or style guides and to leverage code generation of interactive software systems, so that various UI platforms for web, desktop or mobile Applications are supported. Because large parts of the UI are described platform independent instead of platform dependent implementations, re-usability of the UI concept is also improved. However, UI formalization requires the usage of a formal UI description language and a higher level of abstractness compared to concrete UI code. These languages need to be …

Application Of Myo Gesture Recognition Method In Ancient Building Roaming System, Yanping Xue, Xuesong Wang, Zhongke Wu, Xingce Wang, Mingquan Zhou

Journal of System Simulation

Abstract: Museum display many exhibits, but its display way is single. The ancient buildings are displayed by texts and pictures, so it cannot bring a realistic experience to the visitors. Therefore, the ancient architectural scene roaming based on Myo combined with virtual reality is proposed. Kaifeng Tower Park is used as an example to achieve the system. In the design, system gets visitors’ acceleration and gyroscope values of forward, turn, squat and stop by wearing Myo. SVM-based gesture recognition algorithm is designed to complete the classification of feature values. Contrast experiment is designed to verify accuracy and efficiency of …

Twitter And Disasters: A Social Resilience Fingerprint, Benjamin A. Rachunok, Jackson B. Bennett, Roshanak Nateghi

Purdue University Libraries Open Access Publishing Fund

Understanding the resilience of a community facing a crisis event is critical to improving its adaptive capacity. Community resilience has been conceptualized as a function of the resilience of components of a community such as ecological, infrastructure, economic, and social systems, etc. In this paper, we introduce the concept of a “resilience fingerprint” and propose a multi-dimensional method for analyzing components of community resilience by leveraging existing definitions of community resilience with data from the social network Twitter. Twitter data from 14 events are analyzed and their resulting resilience fingerprints computed. We compare the fingerprints between events and show that …

Immersive Virtual Reality Attacks And The Human Joystick, Peter Casey, Ibrahim Baggili, Ananya Yarramreddy

Electrical & Computer Engineering and Computer Science Faculty Publications

This is one of the first accounts for the security analysis of consumer immersive Virtual Reality (VR) systems. This work breaks new ground, coins new terms, and constructs proof of concept implementations of attacks related to immersive VR. Our work used the two most widely adopted immersive VR systems, the HTC Vive, and the Oculus Rift. More specifically, we were able to create attacks that can potentially disorient users, turn their Head Mounted Display (HMD) camera on without their knowledge, overlay images in their field of vision, and modify VR environmental factors that force them into hitting physical objects and …

Reasoning About Ideal Interruptible Moments: A Soft Computing Implementation Of An Interruption Classifier In Free-Form Task Environments, Edward R. Sykes

Publications and Scholarship

Current trends in society and technology make the concept of interruption a central human computer interaction problem. In this work, a novel soft computing implementation for an Interruption Classifier was designed, developed and evaluated that draws from a user model and real-time observations of the user's actions as s/he works on computer-based tasks to determine ideal times to interact with the user. This research is timely as the number of interruptions people experience daily has grown considerably over the last decade. Thus, systems are needed to manage interruptions by reasoning about ideal timings of interactions. This research shows: (1) the …

Computational Modeling Of Facial Response For Detecting Differential Traits In Autism Spectrum Disorders, Manar D. Samad

Electrical & Computer Engineering Theses & Dissertations

This dissertation proposes novel computational modeling and computer vision methods for the analysis and discovery of differential traits in subjects with Autism Spectrum Disorders (ASD) using video and three-dimensional (3D) images of face and facial expressions. ASD is a neurodevelopmental disorder that impairs an individual’s nonverbal communication skills. This work studies ASD from the pathophysiology of facial expressions which may manifest atypical responses in the face. State-of-the-art psychophysical studies mostly employ na¨ıve human raters to visually score atypical facial responses of individuals with ASD, which may be subjective, tedious, and error prone. A few quantitative studies use intrusive sensors on …

Graph-Based Data Visualization In Virtual Reality: A Comparison Of User Experiences, Paul Andrew Sullivan

Master's Theses

Virtual reality is currently in the spotlight, attracting intense scrutiny by the press, technology companies, and consumers. After twenty years of dormancy, the technology has experienced a renaissance following advancements in both hardware and software. Most of these developments, and thus press attention, have been centered on entertainment. However, if the technology is going to continue to grow, it must be useful in a wide variety of situations. This paper explores the effectiveness of visualizing data in virtual reality in comparison to traditional presentations.

In this thesis, we present an approach to displaying data in a virtual reality environment. We …

Comparing Role-Playing Activities In Second Life And Face-To-Face Environments, Fei Gao, Jeongmin J. Noh, Matthew J. Koehler

Fei Gao

This study compared student performances in role-playing activities in both a face-to-face (FTF) environment and a virtual 3D environment, Second Life (SL). We found that students produced a similar amount of communication in the two environments, but the communication styles were different. In SL role-playing activities, students took more conversational turns, but have shorter exchanges compared to the FTF environment. Students generated an equal amount of topic-related concepts in the two environments. They also reported role-playing activities in SL as more interesting and less formal. The educational implications for this study are discussed.

Facilitating Information Retrieval In Social Media User Interfaces, Anthony Costello

Electronic Theses and Dissertations

As the amount of computer mediated information (e.g., emails, documents, multi-media) we need to process grows, our need to rapidly sort, organize and store electronic information likewise increases. In order to store information effectively, we must find ways to sort through it and organize it in a manner that facilitates efficient retrieval. The instantaneous and emergent nature of communications across networks like Twitter makes them suitable for discussing events (e.g., natural disasters) that are amorphous and prone to rapid changes. It can be difficult for an individual human to filter through and organize the large amounts of information that can …

Gesture Based Home Automation For The Physically Disabled, Alexander Hugh Nelson

Graduate Theses and Dissertations

Paralysis and motor-impairments can greatly reduce the autonomy and quality of life of a patient while presenting a major recurring cost in home-healthcare. Augmented with a non-invasive wearable sensor system and home-automation equipment, the patient can regain a level of autonomy at a fraction of the cost of home nurses. A system which utilizes sensor fusion, low-power digital components, and smartphone cellular capabilities can extend the usefulness of such a system to allow greater adaptivity for patients with various needs. This thesis develops such a system as a Bluetooth enabled glove device which communicates with a remote web server to …

Bridging The Gap Between Fun And Fitness: Instructional Techniques And Real-World Applications For Full-Body Dance Games, Emiko Charbonneau

Electronic Theses and Dissertations

Full-body controlled games offer the opportunity for not only entertainment, but education and exercise as well. Refined gameplay mechanics and content can boost intrinsic motivation and keep people playing over a long period of time, which is desirable for individuals who struggle with maintaining a regular exercise program. Within this gameplay genre, dance rhythm games have proven to be popular with game console owners. Yet, while other types of games utilize story mechanics that keep players engaged for dozens of hours, motion-controlled dance games are just beginning to incorporate these elements. In addition, this control scheme is still young, only …

Mac's Fiesta: A Foreign Language Game For The Sifteo Platform, Karina Cordon

Computer Engineering

Mac’s Fiesta is a Spanish learning game designed on the Sifteo gaming platform for children ages four and up. The Sifteo gaming platform provides a new way of interacting with computers by giving users a set of tangible and interactive cubes. Each Sifteo cube has a clickable colored screen and can sense motion and adjacent blocks. Games on this platform are designed to encourage the development of core thinking skills. This project aims to explore how Sifteo cubes can be used for language learning as well as observe how effective they are as educational tools.

Considering Etiquette In The Design Of An Adaptive System, Michael C. Dorneich, Patricia May Ververs, Santosh Mathan, Stephen Whitlow, Caroline C. Hayes

Michael C. Dorneich

In this article, the authors empirically assess the costs and benefits of designing an adaptive system to follow social conventions regarding the appropriateness of interruptions. Interruption management is one area within the larger topic of automation etiquette. The authors tested these concepts in an outdoor environment using the Communications Scheduler, a wearable adaptive system that classifies users' cognitive state via brain and heart sensors and adapts its interactions. Designed to help dismounted soldiers, it manages communications in much the same way as a good administrative assistant. Depending on a combination of message priority, user workload, and system state, it decides …

The Impact Of User Attributes And User Choice In An Agent-Based Environment, Yanghee Kim, Quan Wei

Yanghee Kim

This study examined the impact of learners’ attributes (gender and ethnicity) on their choice of a pedagogical agent and the impact of the attributes and choice on their perceptions of agent affability, task-specific attitudes, task-specific self-efficacy, and learning gains. Participants were 210 high-school male and female, Caucasian and Hispanic students who worked at computer-based algebra integrated with pedagogical agents. The results indicated, first, that students preferentially chose a same-gender agent and a same-ethnicity agent, supporting similarity-attraction theory. Second, males who chose an agent showed more positive attitudes toward working at the learning environment than did males who were assigned to …

Development Of A Cognitive Work Analysis Framework Tutorial Using Systems Modeling Language, Wilfred Henry Wells

Electronic Theses and Dissertations

At the present time, most systems engineers do not have access to cognitive work analysis information or training in terms they can understand. This may lead to a disregard of the cognitive aspect of system design. The impact of this issue is system requirements that do not account for the cognitive strengths and limitations of users. Systems engineers cannot design effective decision support systems without defining cognitive work requirements. In order to improve system requirements, integration of cognitive work requirements into the systems engineering process has to be improved. One option to address this gap is the development of a …

Multi-Touch For General-Purpose Computing An Examination Of Text Entry, Paul David Varcholik

Electronic Theses and Dissertations

In recent years, multi-touch has been heralded as a revolution in humancomputer interaction. Multi-touch provides features such as gestural interaction, tangible interfaces, pen-based computing, and interface customization – features embraced by an increasingly tech-savvy public. However, multi-touch platforms have not been adopted as "everyday" computer interaction devices; that is, multi-touch has not been applied to general-purpose computing. The questions this thesis seeks to address are: Will the general public adopt these systems as their chief interaction paradigm? Can multi-touch provide such a compelling platform that it displaces the desktop mouse and keyboard? Is multi-touch truly the next revolution in human-computer …

Automated Semantic Understanding Of Human Emotions In Writing And Speech, Ricardo A. Calix

LSU Doctoral Dissertations

Affective Human Computer Interaction (A-HCI) will be critical for the success of new technologies that will prevalent in the 21st century. If cell phones and the internet are any indication, there will be continued rapid development of automated assistive systems that help humans to live better, more productive lives. These will not be just passive systems such as cell phones, but active assistive systems like robot aides in use in hospitals, homes, entertainment room, office, and other work environments. Such systems will need to be able to properly deduce human emotional state before they determine how to best interact with …

Episodic Memory Model For Embodied Conversational Agents, Miguel Elvir

Electronic Theses and Dissertations

Embodied Conversational Agents (ECA) form part of a range of virtual characters whose intended purpose include engaging in natural conversations with human users. While works in literature are ripe with descriptions of attempts at producing viable ECA architectures, few authors have addressed the role of episodic memory models in conversational agents. This form of memory, which provides a sense of autobiographic record-keeping in humans, has only recently been peripherally integrated into dialog management tools for ECAs. In our work, we propose to take a closer look at the shared characteristics of episodic memory models in recent examples from the field. …

Comparing Role-Playing Activities In Second Life And Face-To-Face Environments, Fei Gao, Jeongmin J. Noh, Matthew J. Koehler

Visual Communications and Technology Education Faculty Publications

This study compared student performances in role-playing activities in both a face-to-face (FTF) environment and a virtual 3D environment, Second Life (SL). We found that students produced a similar amount of communication in the two environments, but the communication styles were different. In SL role-playing activities, students took more conversational turns, but have shorter exchanges compared to the FTF environment. Students generated an equal amount of topic-related concepts in the two environments. They also reported role-playing activities in SL as more interesting and less formal. The educational implications for this study are discussed.

A Systematic Tool For Deriving Crew Console Layouts, Olu Olofinboba, Robert Demers, Michael C. Dorneich, Chris Hamblin, John Wise

Michael C. Dorneich

This paper describes the use of the Function Allocation Matrix Tool (FAMT) for designing spacecraft cockpit layouts. NASA's Constellation Program intends to return humans to the moon by 2020, followed by exploration to Mars and beyond. The Orion Crew Exploration Vehicle (CEV) will serve as primary vehicle for transporting the crew. Orion will be equipped with a modern 'glass cockpit' that will allow the operators to command and control all of the vehicle's systems via graphics-based displays not unlike those now common in modern flight decks. The design of Orion's displays and controls places an increased emphasis on human-computer interaction …

Communication Blades: Modular Communications For Tangible And Embedded Interfaces, Karun Kumar Kallakuri

LSU Master's Theses

Bladed Tiles is a modular hardware toolkit for building tangible and embedded interface devices. It includes “function blades” and “interaction tiles,” which can provide a flexible, inexpensive, open-ended platform for constructing a wide variety of tangible and embedded interfaces. In this paper, we propose Communication Blades. These are a class of electronic modules with varied computational capabilities for interfacing devices built using bladed tiles toolkit and also for interfacing embedded devices as adapters with external communication networks. These blades provide flexibility by offering the ability to select between different communication technologies and connectivity by providing devices with interoperability over different …

The Contextable: Building And Testing An Intelligent, Context-Aware Kitchen Table, Daniel Matthew Hoopes

Theses and Dissertations

The purpose of this thesis was to design and evaluate The ContexTable, a context-aware system built into a kitchen table. After establishing the current status of the field of context-aware systems and the hurdles and problems being faced, a functioning prototype system was designed and built. The prototype makes it possible to explore established, untested theory and novel solutions to problems faced in the field.

Seven Principles Of Efficient Human Robot Interaction, Michael A. Goodrich, Dan R. Olsen Jr.

Faculty Publications

Advances in robot technology and artificial intelligence have increased the range of robot applications as well as the importance of supporting human interaction with robots and robot teams. Previous work by the authors has highlighted the importance of creating neglect tolerant autonomy and efficient interfaces. In this paper, lessons learned from evaluating neglect tolerance and interface efficiency are compiled into a set of principles for efficient interaction. Emphasis is placed on designing efficient interfaces, but many of the principles require autonomy levels that support the principles. Each principle is illustrated by an example and motivated by citing relevant factors from …

A Technology Assessment Survey For Web Based Higher Education Programs, Earl A. Evans, Susan L. Murray

Engineering Management and Systems Engineering Faculty Research & Creative Works

Advances in communications and computer technology, as well as in human-computer interfaces, have enabled concurrent advances in Web-based education. A number of case studies concerning applications of Web-based education for both distance learning and on-campus programs have been published. Primarily, these studies have focused on individual assessments of the web-based technologies. In contrast, this paper will provide a broad based assessment of applied web-based technology for higher education. This research was conducted via a survey completed by university and college faculty from numerous 4-year institutions. To gain an effective assessment, eleven categories of web-based course delivery tools, such as chatrooms …