Graphics and Human Computer Interfaces Commons^™

A Graph Description Language For Blind Programmers, Dale E. Parson, Genevieve Smith, Andrew Wernicki

Computer Science and Information Technology Faculty

Sighted people often overestimate the usefulness of tactile diagrams for blind people who need to perceive and comprehend entities and relationships in visual structural graphs. Sighted software developers can acquire an overview of the structure of a Unified Modeling Language (UML) design diagram after a quick visual scan. Blind developers using tactile diagrams are impeded by the serial nature of scanning these diagrams with their fingers. However, blind programmers are experts at using text-to-speech screen readers to verbalize lines of existing code at a very rapid rate. Comprehending and debugging code with a screen reader happens extremely quickly because blind …

Toward An Iot-Based Expert System For Heart Disease Diagnosis, Do Thanh Thai, Quang Tran Minh, Phu Huu Phung

Computer Science Faculty Publications

IoT technology has been recently adopted in the healthcare system to collect Electrocardiogram (ECG) signals for heart disease diagnosis and prediction. However, noises in collected ECG signals make the diagnosis and prediction system unreliable and imprecise. In this work, we have proposed a new lightweight approach to removing noises in collected ECG signals to perform precise diagnosis and prediction. First, we have used a revised Sequential Recursive (SR) algorithm to transform the signals into digital format. Then, the digital data is proceeded using a revised Discrete Wavelet Transform (DWT) algorithm to detect peaks in the data to remove noises. Finally, …

Probabilistic Roadmaps For Virtual Camera Pathing With Cinematographic Principles, Katherine Davis

Master's Theses

As technology use increases in the world and inundates everyday life, the visual aspect of technology or computer graphics becomes increasingly important. This thesis presents a system for the automatic generation of virtual camera paths for fly-throughs of a digital scene. The sample scene used in this work is an underwater setting featuring a shipwreck model with other virtual underwater elements such as rocks, bubbles and caustics. The digital shipwreck model was reconstructed from an actual World War II shipwreck, resting off the coast of Malta. Video and sonar scans from an autonomous underwater vehicle were used in a photogrammetry …

Music Times: A Music Learning Game, Emily Woo

Computer Science and Software Engineering

Music Times is a two dimensional educational video game with the purpose of gamifying learning of musical concepts. It has elements of adventure and visual novel games, and incentivizes the player to learn music to explore new levels. It is developed in the Unity game engine, scripted in C#, and targeted for mobile devices. It has six working levels: three lesson levels and three corresponding challenge levels. Each level contains slight differences in visual and aural feel.

Using Intel Realsense Depth Data For Hand Tracking In Unreal Engine 4, Granger Lang

Liberal Arts and Engineering Studies

This project describes how to build a hand tracking method for VR/AR using the raw data from a depth sensing camera.

Feature Learning Via Partial Differential Equation With Applications To Face Recognition, Cong Fang, Zhenyu Zhao, Pan Zhou, Zhouchen Lin

Research Collection School Of Computing and Information Systems

Feature learning is a critical step in pattern recognition, such as image classification. However, most of the existing methods cannot extract features that are discriminative and at the same time invariant under some transforms. This limits the classification performance, especially in the case of small training sets. To address this issue, in this paper we propose a novel Partial Differential Equation (PDE) based method for feature learning. The feature learned by our PDE is discriminative, also translationally and rotationally invariant, and robust to illumination variation. To our best knowledge, this is the first work that applies PDE to feature learning …

Empath-D: Empathetic Design For Accessibility, Kenny Tsu Wei Choo, Rajesh Krishna Balan, Kiat Wee Tan, Archan Misra, Youngki Lee

Research Collection School Of Computing and Information Systems

We describe our vision for Empath-D, our system to enable Empathetic User Interface Design. Our key idea is to leverage Virtual and Augmented Reality (VR / AR) displays to provide an Immersive Reality environment, where developers/designers can emulate impaired interactions by elderly or disabled users while testing the usability of their applications. Our early experiences with the Empath-D prototype show that Empath-D can emulate a cataract vision impairment of the elderly and guide designers to create accessible web pages with less mental workload.

Unsupervised Visual Hashing With Semantic Assistant For Content-Based Image Retrieval, Lei Zhu, Jialie Shen, Liang Xie, Zhiyong Cheng

Research Collection School Of Computing and Information Systems

As an emerging technology to support scalable content-based image retrieval (CBIR), hashing has recently received great attention and became a very active research domain. In this study, we propose a novel unsupervised visual hashing approach called semantic-assisted visual hashing (SAVH). Distinguished from semi-supervised and supervised visual hashing, its core idea is to effectively extract the rich semantics latently embedded in auxiliary texts of images to boost the effectiveness of visual hashing without any explicit semantic labels. To achieve the target, a unified unsupervised framework is developed to learn hash codes by simultaneously preserving visual similarities of images, integrating the semantic …

Active Video Summarization: Customized Summaries Via On-Line Interaction With The User, Ana Garcia Del Molino, Xavier Boix, Joo-Hwee Lim, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

To facilitate the browsing of long videos, automatic video summarization provides an excerpt that represents its content. In the case of egocentric and consumer videos, due to their personal nature, adapting the summary to specific user’s preferences is desirable. Current approaches to customizable video summarization obtain the user’s preferences prior to the summarization process. As a result, the user needs to manually modify the summary to further meet the preferences. In this paper, we introduce Active Video Summarization (AVS), an interactive approach to gather the user’s preferences while creating the summary. AVS asks questions about the summary to update it …

Static Human Detection And Scenario Recognition Via Wearable Thermal Sensing System, Qingquan Sun, Ju Shen, Haiyan Qiao, Xinlin Huang, Chen Chen, Fei Hu

Computer Science Faculty Publications

Conventional wearable sensors are mainly used to detect the physiological and activity information of individuals who wear them, but fail to perceive the information of the surrounding environment. This paper presents a wearable thermal sensing system to detect and perceive the information of surrounding human subjects. The proposed system is developed based on a pyroelectric infrared sensor. Such a sensor system aims to provide surrounding information to blind people and people with weak visual capability to help them adapt to the environment and avoid collision. In order to achieve this goal, a low-cost, low-data-throughput binary sampling and analyzing scheme is …

Older Adult Health: National Library Of Medicine Resources For Health Care Providers And For Patients And Families, Elizabeth Dyer, Barbara Swartzlander, Marilyn R. Gugliucci, Laura Taylor

Library Services Faculty Publications

This list of resources was designed to complement a project funded by the National Network of Libraries of Medicine (NN/LM) New England Region (NER) entitled “Empathy Learned Through an Extended Medical Education Virtual Reality Project." The project used a virtual reality (VR) experience for 1st year medical students developed by Embodied Labs. The interactive “Alfred Lab” immerses users in the story of a 74-year-old patient who has macular degeneration and hearing loss, allowing users to experience these conditions from the patient’s perspective as he interacts with his family and doctor.

Unboxing Cluster Heatmaps, Sophie J. Engle, S. Whalen, Alark Joshi, K. Pollard

Computer Science

Background: Cluster heatmaps are commonly used in biology and related fields to reveal hierarchical clusters in data matrices. This visualization technique has high data density and reveal clusters better than unordered heatmaps alone. However, cluster heatmaps have known issues making them both time consuming to use and prone to error. We hypothesize that visualization techniques without the rigid grid constraint of cluster heatmaps will perform better at clustering-related tasks.

Results: We developed an approach to “unbox” the heatmap values and embed them directly in the hierarchical clustering results, allowing us to use standard hierarchical visualization techniques as alternatives …

A System To Monitor Cognitive Workload In Naturalistic High-Motion Environments, Bethany K. Bracken, Seth Elkin-Frankston, Noa Palmon, Michael Farry, Blaise De B Frederick

H-Workload 2017: Models and Applications (Works in Progress)

Across many careers, individuals face alternating periods of high and low attention and cognitive workload can impair cognitive function and undermine job performance. We have designed and are developing an unobtrusive system to Monitor, Extract, and Decode Indicators of Cognitive Workload (MEDIC) in naturalistic, high-motion environments. MEDIC is designed to warn individuals, teammates, or supervisors when steps should be taken to augment cognitive readiness. We first designed and manufactured a forehead sensor device that includes a custom fNIRS sensor and a three-axis accelerometer designed to be mounted on the inside of a baseball cap or headband, or standard issue gear …

Towards A Not Obtrusive Low Cost Biosystem To Assess Risk Perception In Workplace Through Stress Detection, Emanuele Bellini, Serena Benevenuti, Chiara Batistini

H-Workload 2017: Models and Applications (Works in Progress)

The main aim of the article is to build a method to assess risk perception in real time in order to early detect and prevent risk behaviors and possible human errors. To this end, the relation between mental workload and stress as critical factors affecting risk perception has been investigated. In particular the mental-physical activation generated by an increment of the workload has the effect of reducing the resources needed to perceive risk increasing the worker vulnerability. The complexity of the stress phenomenon suggested the adoption of an integrated view. The Functional Model has been adopted to for its holistic …

Hindsight: Encouraging Exploration Through Direct Encoding Of Personal Interaction History, Mi Feng, Cheng Deng, Evan M. Peck, Lane Harrison

Faculty Journal Articles

Physical and digital objects often leave markers of our use. Website links turn purple after we visit them, for example, showing us information we have yet to explore. These “footprints” of interaction offer substantial benefits in information saturated environments - they enable us to easily revisit old information, systematically explore new information, and quickly resume tasks after interruption. While applying these design principles have been successful in HCI contexts, direct encodings of personal interaction history have received scarce attention in data visualization. One reason is that there is little guidance for integrating history into visualizations where many visual channels are …

A Pilot Study Into Bio-Behavioural Measurements On Air Traffic Controllers In Remote Tower Operations, Tanja Bos, Rolf Zon, Eszter Furedi, Dezso Dudas, Daniel Rohacs

H-Workload 2017: Models and Applications (Works in Progress)

What is the impact of shifting to remote tower operations on the Air Traffic Controller? In the joint HungaroControl-Netherlands Aerospace Centre NLR pilot project an assessment of bio-behaviour on three air traffic controllers was made in a remote tower and conventional tower. The research is motivated by HungaroControl’s plans in shifting to remote tower operations at Budapest airport in the upcoming years. This pilot project is considered a feasibility study to investigate if an eye tracker and a heart rate sensor can be used to derive workload, the controllers’ division of attention over information elements, and scanning strategies in two …

A Systems Approach To Predicting And Measuring Workload In Rail Traffic Management Systems, Joanna Evans

H-Workload 2017: Models and Applications (Works in Progress)

The introduction of systems such as Traffic Management (TM) will result in a number of changes in how the railway is managed for operations and maintenance staff such as, an increase in collaborative working styles and shared responsibilities. In order to react to these changing operational demands and user needs, TM workstation designs need to have greater flexibility and be configurable to support the information requirements for each specific role as well as support each role during different scenarios. Although this flexibility in system design has the potential to enhance performance, it increases the complexity of measuring operator workload. The …

Correcting Pedestrian Dead Reckoning With Monte Carlo Localization Boxed For Indoor Navigation, Akira T. Murphy

Honors Theses

Localization of phones is a ubiquitous part of the modern mobile electronics landscape. However, there are many situations where the current method of networked localization fails. A Pedestrian Dead Reckoning System where the location of the user is calculated by counting the steps and direction of the user was implemented as an iOS app with python for data analysis. A novel algorithm for wireless sensor localization using Ad-Hoc Bluetooth networks was proposed. A small experiment was performed proving that the system is nearly equal to state of the art algorithms.

Cse: U: Mixed-Initiative Personal Assistant Agents, Joshua W. Buck, Saverio Perugini, Tam Nguyen

Computer Science Faculty Publications

Specification and implementation of flexible human-computer dialogs is challenging because of the complexity involved in rendering the dialog responsive to a vast number of varied paths through which users might desire to complete the dialog. To address this problem, we developed a toolkit for modeling and implementing task-based, mixed-initiative dialogs based on metaphors from lambda calculus. Our toolkit can automatically operationalize a dialog that involves multiple prompts and/or sub-dialogs, given a high-level dialog specification of it. The use of natural language with the resulting dialogs makes the flexibility in communicating user utterances commensurate with that in dialog completion paths—an aspect …

Building A Foundation For Digital Animation, Paulynne Castillo

Angelo King Institute for Economic and Business Studies (AKI)

An analysis of the Philippine digital animation sub-sector reveals that the key areas on which resources should be focused to move up the industry’s value chain are: 1) recruiting, developing, and retaining talent; 2) financing the cultivation of original content; and 3) expanding and improving the country’s information communication technology (ICT) infrastructure.

Evolving Art: Modifying Context Free Art With A Genetic Algorithm, Marina Kent

Scripps Senior Theses

Context Free Design Grammar (CFDG) is a programming language for defining recursive structures that can be used to create art. I use CFDG as a design space for genetic programming, experimenting with various options for crossover, mutation, and fitness. In this exploratory work, multiple generations are manually assessed to determine the usefulness of the mutation strategies and fitness functions. I find that simple value mutation and fitness that alters general program structure is not enough to produce an increase of interesting images in CFDG. I discuss these findings as well as future avenues of inquiry for genetic programming in artistic …

Visualizing Multidimensional Data With General Line Coordinates And Pareto Optimization, Jacob Brown

All Master's Theses

These results will show that the use of Linear General Line Coordinates (GLC-L) can visualize multidimensional data better than typical methods, such as Parallel Coordinates (PC). The results of using GLC-L will display visuals with less clutter than PC and be easier to see changes from one graph to the next. Visualizing the Pareto Frontier with GLC-L allows n-D data to be viewed at once, compared to typical methods that are limited to 2 or 3 objectives at a time. This method details the process of selecting a ”best” case, from a group of equals in the Pareto Subset and …

Creating A Game With Procedural Generation, Kirsten Baker

Williams Honors College, Honors Research Projects

In today’s world, gaming is ever growing in popularity. The amount of games is forever increasing and there are many genres of games to suit any player’s desires. I have created a game that generates explorable mazes. Originally written in Java as a 2D game, I translated it into C# and utilized the Unity game engine to make it 3D. The randomly generated mazes would allow players to have endless fun navigating through the different levels and customizing it to fit their preferences.

Iphone Swift 3 Development For “What’S Good?”, Cameron Reilly

Williams Honors College, Honors Research Projects

Cameron Reilly

Major: Computer Science Management Track

Project Sponsor: Michael Collard

What’s Good?

“What’s Good?” is an iOS application that uses augmented reality to present information about nearby restaurants to the user in real time. Users can hold up their device and look through their camera to see information presented above the restaurant including it’s name, average cost, and average review rating. The user’s location is gathered and used in tandem with the Zomato API to select what data display as those dining locations are currently near the user. Using the gyroscope, compass, and accelerometer to tell the exact orientation …

Game Collection Development And Marketing, Todd R. Locker Jr

Williams Honors College, Honors Research Projects

The reasons for completing this project include expanding my knowledge of programming features and methodology, and to learn about game platform development and design. This was done by using the Java programming language to create a game platform which can host a variable number of games. One of the most notable features that was utilized is the JavaFX platform. Much programming experience was gained from this platform and all the features and methods it provides for customization. The application that was produced is graphical user interface based and created with event-driven programming. The result of this project was a custom …

Designing An Intervention For Novice Programmers Based On Meaningful Gamification: An Expert Evaluation, Jenilyn L. Agapito, Ma. Mercedes T. Rodrigo

Department of Information Systems & Computer Science Faculty Publications

Gamification is defined as the addition of game-like elements and mechanics to non-game contexts to encourage certain desired behaviors. It is becoming a popular classroom intervention used in computer science instruction, including CS1, the first course computer science students take. It is being operationalized to enhance students' learning experience and achievement. However, existing studies have mostly implemented reward-based game elements which have resulted to contrasting behaviors among the students. Meaningful gamification, characterized as the use of game design elements to encourage users build internal motivation to behave in a certain way, is contended to be a more effective approach. The …

Real-Time Capture And Rendering Of Physical Scene With An Efficiently Calibrated Rgb-D Camera Network, Po-Chang Su

Theses and Dissertations--Electrical and Computer Engineering

From object tracking to 3D reconstruction, RGB-Depth (RGB-D) camera networks play an increasingly important role in many vision and graphics applications. With the recent explosive growth of Augmented Reality (AR) and Virtual Reality (VR) platforms, utilizing camera RGB-D camera networks to capture and render dynamic physical space can enhance immersive experiences for users. To maximize coverage and minimize costs, practical applications often use a small number of RGB-D cameras and sparsely place them around the environment for data capturing. While sparse color camera networks have been studied for decades, the problems of extrinsic calibration of and rendering with sparse RGB-D …

Grasping The Void: Immersion Tactics Using Gesture Controlled Physics Interaction Systems In Virtual Reality, Avery Rapson

Senior Independent Study Theses

This thesis uses the HTC Vive in Unity to compare two different types of object interaction systems in order to determine the effectiveness of physics based interaction systems in a virtual environment. The research problem that motivates this project is the fact that there is no standardized method for defining successful object interaction techniques in VR. There are numerous interaction techniques in VR that fall short of simulating realistic object interaction. This project explores a physics based interaction system and examines how effective it is by comparing it to a non-physics based system. A model house with various interactable objects …

Big Data Visualization Platform For Mixed Reality, Aliakbar Panahi

Theses and Dissertations

The visualization of data helps to provide faster and deeper insight into the data. In this work, a system for visualizing and analyzing big data in an interactive mixed reality environment is proposed. Such a system can be used for representing different types of data such as temporal, geospatial, network graph, and high dimensional. Also, an implementation of this system for four different data types are created. The data types include network data, volumetric data, high dimensional, and spectral data for different mixed reality devices such as Microsoft HoloLens, Oculus Rift, Samsung Gear VR, and Android ARCore were created. It …

Distress And Worry As Mediators In The Relationship Between Psychosocial Risks And Upper Body Musculosketal Complaints In Highly Automated Manufacturing, Fiona Wixted, Leonard O'Sullivan

H-Workload 2017: Models and Applications (Works in Progress)

As a result of an upward trend in automation, the requirement for supervisory monitoring and consequently, cognitive demand has increased in automated manufacturing. The incidence of musculoskeletal disorders has also increased in the manufacturing sector. A model was developed based on survey data to test if distress and worry mediate the relationship between psychosocial factors (job control, cognitive demand, social isolation and skill discretion), stress states and upper body musculoskeletal complaints in highly automated manufacturing companies (n=235). Cognitive demand was shown to be related to higher distress in employees. The data raise the question about the link between job control …