Graphics and Human Computer Interfaces Commons^™

A Graph Description Language For Blind Programmers, Dale E. Parson, Genevieve Smith, Andrew Wernicki

Computer Science and Information Technology Faculty

Sighted people often overestimate the usefulness of tactile diagrams for blind people who need to perceive and comprehend entities and relationships in visual structural graphs. Sighted software developers can acquire an overview of the structure of a Unified Modeling Language (UML) design diagram after a quick visual scan. Blind developers using tactile diagrams are impeded by the serial nature of scanning these diagrams with their fingers. However, blind programmers are experts at using text-to-speech screen readers to verbalize lines of existing code at a very rapid rate. Comprehending and debugging code with a screen reader happens extremely quickly because blind …

Toward An Iot-Based Expert System For Heart Disease Diagnosis, Do Thanh Thai, Quang Tran Minh, Phu Huu Phung

Computer Science Faculty Publications

IoT technology has been recently adopted in the healthcare system to collect Electrocardiogram (ECG) signals for heart disease diagnosis and prediction. However, noises in collected ECG signals make the diagnosis and prediction system unreliable and imprecise. In this work, we have proposed a new lightweight approach to removing noises in collected ECG signals to perform precise diagnosis and prediction. First, we have used a revised Sequential Recursive (SR) algorithm to transform the signals into digital format. Then, the digital data is proceeded using a revised Discrete Wavelet Transform (DWT) algorithm to detect peaks in the data to remove noises. Finally, …

Probabilistic Roadmaps For Virtual Camera Pathing With Cinematographic Principles, Katherine Davis

Master's Theses

As technology use increases in the world and inundates everyday life, the visual aspect of technology or computer graphics becomes increasingly important. This thesis presents a system for the automatic generation of virtual camera paths for fly-throughs of a digital scene. The sample scene used in this work is an underwater setting featuring a shipwreck model with other virtual underwater elements such as rocks, bubbles and caustics. The digital shipwreck model was reconstructed from an actual World War II shipwreck, resting off the coast of Malta. Video and sonar scans from an autonomous underwater vehicle were used in a photogrammetry …

Music Times: A Music Learning Game, Emily Woo

Computer Science and Software Engineering

Music Times is a two dimensional educational video game with the purpose of gamifying learning of musical concepts. It has elements of adventure and visual novel games, and incentivizes the player to learn music to explore new levels. It is developed in the Unity game engine, scripted in C#, and targeted for mobile devices. It has six working levels: three lesson levels and three corresponding challenge levels. Each level contains slight differences in visual and aural feel.

Using Intel Realsense Depth Data For Hand Tracking In Unreal Engine 4, Granger Lang

Liberal Arts and Engineering Studies

This project describes how to build a hand tracking method for VR/AR using the raw data from a depth sensing camera.

Feature Learning Via Partial Differential Equation With Applications To Face Recognition, Cong Fang, Zhenyu Zhao, Pan Zhou, Zhouchen Lin

Research Collection School Of Computing and Information Systems

Feature learning is a critical step in pattern recognition, such as image classification. However, most of the existing methods cannot extract features that are discriminative and at the same time invariant under some transforms. This limits the classification performance, especially in the case of small training sets. To address this issue, in this paper we propose a novel Partial Differential Equation (PDE) based method for feature learning. The feature learned by our PDE is discriminative, also translationally and rotationally invariant, and robust to illumination variation. To our best knowledge, this is the first work that applies PDE to feature learning …

Unsupervised Visual Hashing With Semantic Assistant For Content-Based Image Retrieval, Lei Zhu, Jialie Shen, Liang Xie, Zhiyong Cheng

Research Collection School Of Computing and Information Systems

As an emerging technology to support scalable content-based image retrieval (CBIR), hashing has recently received great attention and became a very active research domain. In this study, we propose a novel unsupervised visual hashing approach called semantic-assisted visual hashing (SAVH). Distinguished from semi-supervised and supervised visual hashing, its core idea is to effectively extract the rich semantics latently embedded in auxiliary texts of images to boost the effectiveness of visual hashing without any explicit semantic labels. To achieve the target, a unified unsupervised framework is developed to learn hash codes by simultaneously preserving visual similarities of images, integrating the semantic …

Empath-D: Empathetic Design For Accessibility, Kenny Tsu Wei Choo, Rajesh Krishna Balan, Kiat Wee Tan, Archan Misra, Youngki Lee

Research Collection School Of Computing and Information Systems

We describe our vision for Empath-D, our system to enable Empathetic User Interface Design. Our key idea is to leverage Virtual and Augmented Reality (VR / AR) displays to provide an Immersive Reality environment, where developers/designers can emulate impaired interactions by elderly or disabled users while testing the usability of their applications. Our early experiences with the Empath-D prototype show that Empath-D can emulate a cataract vision impairment of the elderly and guide designers to create accessible web pages with less mental workload.

Active Video Summarization: Customized Summaries Via On-Line Interaction With The User, Ana Garcia Del Molino, Xavier Boix, Joo-Hwee Lim, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

To facilitate the browsing of long videos, automatic video summarization provides an excerpt that represents its content. In the case of egocentric and consumer videos, due to their personal nature, adapting the summary to specific user’s preferences is desirable. Current approaches to customizable video summarization obtain the user’s preferences prior to the summarization process. As a result, the user needs to manually modify the summary to further meet the preferences. In this paper, we introduce Active Video Summarization (AVS), an interactive approach to gather the user’s preferences while creating the summary. AVS asks questions about the summary to update it …

Static Human Detection And Scenario Recognition Via Wearable Thermal Sensing System, Qingquan Sun, Ju Shen, Haiyan Qiao, Xinlin Huang, Chen Chen, Fei Hu

Computer Science Faculty Publications

Conventional wearable sensors are mainly used to detect the physiological and activity information of individuals who wear them, but fail to perceive the information of the surrounding environment. This paper presents a wearable thermal sensing system to detect and perceive the information of surrounding human subjects. The proposed system is developed based on a pyroelectric infrared sensor. Such a sensor system aims to provide surrounding information to blind people and people with weak visual capability to help them adapt to the environment and avoid collision. In order to achieve this goal, a low-cost, low-data-throughput binary sampling and analyzing scheme is …

Older Adult Health: National Library Of Medicine Resources For Health Care Providers And For Patients And Families, Elizabeth Dyer, Barbara Swartzlander, Marilyn R. Gugliucci, Laura Taylor

Library Services Faculty Publications

This list of resources was designed to complement a project funded by the National Network of Libraries of Medicine (NN/LM) New England Region (NER) entitled “Empathy Learned Through an Extended Medical Education Virtual Reality Project." The project used a virtual reality (VR) experience for 1st year medical students developed by Embodied Labs. The interactive “Alfred Lab” immerses users in the story of a 74-year-old patient who has macular degeneration and hearing loss, allowing users to experience these conditions from the patient’s perspective as he interacts with his family and doctor.

Unboxing Cluster Heatmaps, Sophie J. Engle, S. Whalen, Alark Joshi, K. Pollard

Computer Science

Background: Cluster heatmaps are commonly used in biology and related fields to reveal hierarchical clusters in data matrices. This visualization technique has high data density and reveal clusters better than unordered heatmaps alone. However, cluster heatmaps have known issues making them both time consuming to use and prone to error. We hypothesize that visualization techniques without the rigid grid constraint of cluster heatmaps will perform better at clustering-related tasks.

Results: We developed an approach to “unbox” the heatmap values and embed them directly in the hierarchical clustering results, allowing us to use standard hierarchical visualization techniques as alternatives …

Petrochemical Plant Console Operator Workload:The Issues, David A. Strobhar

H-Workload 2017: Models and Applications (Works in Progress)

The console operators of certain petrochemical processes must maintain high levels of performance during process upsets or endanger personnel safety and the environment. Mismanagement of an upset can result in explosions, fires, and the release of hazardous chemicals to the environment. The change in workload from steady state to upset operation is significant, with alarms and control changes that are of an order of magnitude. This paper describes the state of console activity in process plants, particularly the increase with key upsets. Quantitative data on the nature of the console operator’s position, its workload during normal operation, and the requirements …

A System To Monitor Cognitive Workload In Naturalistic High-Motion Environments, Bethany K. Bracken, Seth Elkin-Frankston, Noa Palmon, Michael Farry, Blaise De B Frederick

H-Workload 2017: Models and Applications (Works in Progress)

Across many careers, individuals face alternating periods of high and low attention and cognitive workload can impair cognitive function and undermine job performance. We have designed and are developing an unobtrusive system to Monitor, Extract, and Decode Indicators of Cognitive Workload (MEDIC) in naturalistic, high-motion environments. MEDIC is designed to warn individuals, teammates, or supervisors when steps should be taken to augment cognitive readiness. We first designed and manufactured a forehead sensor device that includes a custom fNIRS sensor and a three-axis accelerometer designed to be mounted on the inside of a baseball cap or headband, or standard issue gear …

Smart Workload Balancing, Ferdinand Coster

H-Workload 2017: Models and Applications (Works in Progress)

The cognitive workload of operators working with automated systems should neither be too high nor too low. A static level of automation is unable to cope with systems that produce large fluctuations in cognitive workload, therefore a method for adaptive automation is proposed that could balance workload by intelligently choosing what to automate and when. To this end the concept of the Cognitive Workload Value factor is introduced, which takes into account both workload and situation awareness. This initial work introduces a possible framework for categorizing and using different workload and situation awareness measures.

Towards A Not Obtrusive Low Cost Biosystem To Assess Risk Perception In Workplace Through Stress Detection, Emanuele Bellini, Serena Benevenuti, Chiara Batistini

H-Workload 2017: Models and Applications (Works in Progress)

The main aim of the article is to build a method to assess risk perception in real time in order to early detect and prevent risk behaviors and possible human errors. To this end, the relation between mental workload and stress as critical factors affecting risk perception has been investigated. In particular the mental-physical activation generated by an increment of the workload has the effect of reducing the resources needed to perceive risk increasing the worker vulnerability. The complexity of the stress phenomenon suggested the adoption of an integrated view. The Functional Model has been adopted to for its holistic …

Reducing Peak Workload In The Cockpit: A Human In The Loop Simulation Evaluating New Runway Selection Tool, Tanja Bos, Rolf Zon, Wilfred Rouwhorst

H-Workload 2017: Models and Applications (Works in Progress)

In efforts to increase safety and reduce peak workload situations in the cockpit, a tool with a different interaction style was developed for use in case of a runway change instructed by Air Traffic Control during approach. In an experiment a workload comparison was made between the new tool and the conventional cockpit. Workload was measured by means of a self-rating after each experiment run, as well as eye blink frequency during each run. Results show that the self-rated workload decreases with the new tool for one of the two crew members and the blink frequency suggests a workload decrease …

Managing Operator Mental Workload With Standards Based Decision Support, Maurice Wilkins

H-Workload 2017: Models and Applications (Works in Progress)

H-Workload 2017: The first international symposium on human mental workload, Dublin Institute of Technology, Dublin, Ireland, June 28-30.

Towards A Continuous Assessment Of Cognitive Workload For Smartphone Multitasking Users, Angel Jimenez-Molina, Hernan Lira

H-Workload 2017: Models and Applications (Works in Progress)

The intermeshing of Smartphone interactions and daily activities depletes the availability of cognitive resources. This excessive demand may lead to several undesirable cognitive states, which can be avoided by continuously assessing the user cognitive workload. Recently, many attempts have emerged to assess this workload by using psycho physiological signals. This paper provides evidence that it is possible to train models that accurately identify in short time windows such cognitive workload by processing heart rate and blood oxygen saturation signals. This assessment could be applied in Smartphone notification delivery, interface adaptations or cognitive capabilities evaluation.

Facing Human Workload: The Resilient Ego: A Psychoanalytic Point Of View, Glauco Maria Genga, Maria Gabriella Pediconii

H-Workload 2017: Models and Applications (Works in Progress)

The paper aims to show new connections among Human Factors, Human Workload and Resilience. We intend: 1) to highlight the role of subject in facing the human workload, inflected as demanding tasks and emergency situations; 2) to show how psychoanalysis can provide novel insights, not only into human errors, but also into human resilience. They have a common denominator, at least in part: the role of subjective contributions even in demanding situations. Human workload includes a work for satisfaction. We recall also the case study of US Airways Flight 1549 water landing (the so called “Miracle on the Hudson”), which …

System Identification Of Motion Artifact: Noise In Eeg Headsets From Locomotion, Kaela Shea, James Tung

H-Workload 2017: Models and Applications (Works in Progress)

Fall prevention for geriatric populations is a growing concern among clinicians and researchers due to severe risk of morbidity and loss of independence. Emerging evidence has demonstrated that mental workload while walking influences gait stability and the risk for falling. Electroencephalography (EEG) presents a potential method to provide objective measures of mental workload, particularly during daily activities. Noise introduced to the EEG signal during motion, however, is restrictive. The study presented in the following paper isolates EEG signal noise attained from gait for a commercially accessible EEG system, the "Emotiv" Time and spectral system identification techniques were applied to model …

Online Measuring Of Available Resources, Enrique Munoz-De-Escalona, José Juan Canas

H-Workload 2017: Models and Applications (Works in Progress)

This paper present a proposal for measuring available mental resources during the accomplishment of a task. Our proposal consists in measuring emotions provoked by perceived self-efficacy in the execution of the task. Self-efficacy is one of the most important factors that affect the resources that a person puts at the disposal of the execution of the task. When a person perceives that he/she is not being effective he/she will activate more resources to improve his performance. This self-efficacy will be reflected in the emotions that the person experiences. A good efficacy will provoke positive emotions and a bad efficacy negative …

A Validated Description Of How Crew Manage Flight Operations For Two-Pilot And Reduced Crew Operations, Nick Mcdonnell, Alison Kay, Margaret Ryan, Rabea Morrison, Rolf Zon

H-Workload 2017: Models and Applications (Works in Progress)

This research provides a rich validated description of how crew manage workload for both two-pilot and reduced crew operations. It outlines flight operations modelling, operational narratives, requirements and scenarios validated with expert advisers from the EU-FP7 ACROSS Project. The crew are considered to be the managers of the operation who receive integrated technical support to help them manage flight operations across of three configurations i.e. 1) standard two-crew configuration, 2) reduced crew under normal operations 3) reduced-crew under non-normal operations developed within the FP7 EU-funded ACROSS (Advanced Cockpit for the Reduction Of Stress and Workload) project.

Online Health Recommendation System: A Social Support Perspective, Tareq Nasralah, Cherie Noteboom, Abdullah Wahbeh, Mohammad Aref Al-Ramahi

Research & Publications

Online Health Communities (OHC) aim to support patients through offer them the opportunities to exchange support with others. However, patients have difficulties and problems locating expertise within the online health communities. In this regard, this study aims to create a patient recommender system to help users locate those with relevant experience and similar health status. Specifically, we aim to leverage the type of online social support users seek to determine the patient health status to build a patient status prediction model. Building the model will help create a peer recommendation system for online support group members to easily locate peers …

Distress And Worry As Mediators In The Relationship Between Psychosocial Risks And Upper Body Musculosketal Complaints In Highly Automated Manufacturing, Fiona Wixted, Leonard O'Sullivan

H-Workload 2017: Models and Applications (Works in Progress)

As a result of an upward trend in automation, the requirement for supervisory monitoring and consequently, cognitive demand has increased in automated manufacturing. The incidence of musculoskeletal disorders has also increased in the manufacturing sector. A model was developed based on survey data to test if distress and worry mediate the relationship between psychosocial factors (job control, cognitive demand, social isolation and skill discretion), stress states and upper body musculoskeletal complaints in highly automated manufacturing companies (n=235). Cognitive demand was shown to be related to higher distress in employees. The data raise the question about the link between job control …

A Workload-Centered Perspective On Reduced Crew Operations In Commercial Aviation, Daniela Schmid

H-Workload 2017: Models and Applications (Works in Progress)

Mental workload of a pilot, in short workload, depends on various characteristics of different accumulated tasks on the flight deck. Exogenous task demands and endogenous supply of attentional or information processing resources determine workload [1]. Performance is expect to drop if the demand exceeds the available resources of the pilot. Expertise and experience modulate the endogenous sup- ply of resources like perceiving, updating memory, planing, making a decision, and executing and processing a response. Subsequently, workload manifests in performance variables, subjective experience, and physiological parameters [2]. This is how we can summarize workload very brie y to introduce a model …

A Pilot Study Into Bio-Behavioural Measurements On Air Traffic Controllers In Remote Tower Operations, Tanja Bos, Rolf Zon, Eszter Furedi, Dezso Dudas, Daniel Rohacs

H-Workload 2017: Models and Applications (Works in Progress)

What is the impact of shifting to remote tower operations on the Air Traffic Controller? In the joint HungaroControl-Netherlands Aerospace Centre NLR pilot project an assessment of bio-behaviour on three air traffic controllers was made in a remote tower and conventional tower. The research is motivated by HungaroControl’s plans in shifting to remote tower operations at Budapest airport in the upcoming years. This pilot project is considered a feasibility study to investigate if an eye tracker and a heart rate sensor can be used to derive workload, the controllers’ division of attention over information elements, and scanning strategies in two …

A Systems Approach To Predicting And Measuring Workload In Rail Traffic Management Systems, Joanna Evans

H-Workload 2017: Models and Applications (Works in Progress)

The introduction of systems such as Traffic Management (TM) will result in a number of changes in how the railway is managed for operations and maintenance staff such as, an increase in collaborative working styles and shared responsibilities. In order to react to these changing operational demands and user needs, TM workstation designs need to have greater flexibility and be configurable to support the information requirements for each specific role as well as support each role during different scenarios. Although this flexibility in system design has the potential to enhance performance, it increases the complexity of measuring operator workload. The …

Cse: U: Mixed-Initiative Personal Assistant Agents, Joshua W. Buck, Saverio Perugini, Tam Nguyen

Computer Science Faculty Publications

Specification and implementation of flexible human-computer dialogs is challenging because of the complexity involved in rendering the dialog responsive to a vast number of varied paths through which users might desire to complete the dialog. To address this problem, we developed a toolkit for modeling and implementing task-based, mixed-initiative dialogs based on metaphors from lambda calculus. Our toolkit can automatically operationalize a dialog that involves multiple prompts and/or sub-dialogs, given a high-level dialog specification of it. The use of natural language with the resulting dialogs makes the flexibility in communicating user utterances commensurate with that in dialog completion paths—an aspect …

Visualizing Multidimensional Data With General Line Coordinates And Pareto Optimization, Jacob Brown

All Master's Theses

These results will show that the use of Linear General Line Coordinates (GLC-L) can visualize multidimensional data better than typical methods, such as Parallel Coordinates (PC). The results of using GLC-L will display visuals with less clutter than PC and be easier to see changes from one graph to the next. Visualizing the Pareto Frontier with GLC-L allows n-D data to be viewed at once, compared to typical methods that are limited to 2 or 3 objectives at a time. This method details the process of selecting a ”best” case, from a group of equals in the Pareto Subset and …