Graphics and Human Computer Interfaces Commons^™

Where Did The Time Go?, John C. Viaud, Bilal Abdulmajid, Vitali Surmach, Jia Yanxia

Capstone Showcase

Study shows that most people spend a full quarter of their active hours on their mobile device which can take a serious toll on our productivity as well as our mental and physical well-being. We created an Android app that is able to track and visualize phone usage patterns to help user establish awareness of how much and in what ways they use their Android devices. Unlike currently existing phone time management apps, such as Apple’s ScreenTime and Google’s new Digital Wellbeing, our app provides functionalities to encourage off-phone time and personal goal management.

Needfinding, Devorah Kletenik

Open Educational Resources

This activity guides students through the process needfinding to identify areas of need for their creation of a technology for the "public good." Students will conduct contextual inquiry to identify the needs of their target audience.

Personas, Scenarios And Storyboards, Devorah Kletenik

Open Educational Resources

This activity guides students towards the creation of personas, scenarios and storyboards for a product/website that they are creating.

Public Interest Technology: Coding For The Public Good, Devorah Kletenik

Open Educational Resources

These slides are used to guide a discussion with students introducing them to the notion of public interest technology and coding for the public good. The lesson is intended to spark a discussion with students about different sorts of technology and their societal ramifications.

Accessibility: The Whys And The Hows, Devorah Kletenik

Open Educational Resources

This presentation introduces Computer Science students to the notion of accessibility: developing software for people with disabilities. This lesson provides a discussion of why accessibility is important (including the legal, societal and ethical benefits) as well as an overview of different types of impairments (visual, auditory, motor, neurological/cognitive) and how developers can make their software accessible to users with those disabilities. This lesson includes videos and links to readings and tutorials for students.

Coding For The Public Good: Front-End Website Design And Development, Devorah Kletenik

Open Educational Resources

This activity helps student design and develop a front-end of a website, from wireframes through HTML/CSS/Javascript. It includes design questions for students, including the invocation of Ben Schneiderman's eight golden rules for interface design.

Note: this activity assumes prior knowledge of web development. Since this activity is designed for an HCI course, with a focus on interface design, students are not expected to create a back-end for it. This activity can obviously be modified for a full-stack experience.

Accessibility Evaluation, Devorah Kletenik

Open Educational Resources

This activity guides students through the evaluation of a website that they have created to see if it is accessible for users with disabilities. Students will simulate a number of different disabilities (e.g. visual impairments, color blindness, auditory impairments, motor impairments) to see if their website is accessible; they will also use automated W3 and WAVE tools to evaluate their sites. Students will consider the needs of users with disabilities by creating a persona and scenario of a user with disabilities interacting with their site. Finally, students will write up recommendations to change their site and implement the changes.

Toward Efficient Automation Of Interpretable Machine Learning Boosting, Nathan Neuhaus

All Master's Theses

Developing efficient automated methods for Interpretable Machine Learning (IML) is an important and long-term goal in the field of Artificial Intelligence. Currently the Machine Learning landscape is dominated by Neural Networks (NNs) and Support Vector Machines (SVMs), models which are often highly accurate. Despite high accuracy, such models are essentially “black boxes” and therefore are too risky for situations like healthcare where real lives are at stake. In such situations, so called “glass-box” models, such as Decision Trees (DTs), Bayesian Networks (BNs), and Logic Relational (LR) models are often preferred, however can succumb to accuracy limitations. Unfortunately, having to choose …

Android Game, Ryan Weston

Williams Honors College, Honors Research Projects

The purpose of this project was to create an endless runner game for Android coded in Java and XML and developed in Android Studio. In the game, the player controls a frog that jumps from lily pad to lily pad to avoid logs moving toward the player. The player must also maneuver the lily pads as they can randomly disappear. There are three difficulties in the game that vary the disappearance rate of lily pads as well as the frequency and acceleration rate of the log obstacles. The game also has a scoring system and saves the high score locally …

Navigating Immersive And Interactive Vr Environments With Connected 360° Panoramas, Samuel Cosgrove

Electronic Theses and Dissertations, 2020-2023

Emerging research is expanding the idea of using 360-degree spherical panoramas of real-world environments for use in "360 VR" experiences beyond video and image viewing. However, most of these experiences are strictly guided, with few opportunities for interaction or exploration. There is a desire to develop experiences with cohesive virtual environments created with 360 VR that allow for choice in navigation, versus scripted experiences with limited interaction. Unlike standard VR with the freedom of synthetic graphics, there are challenges in designing appropriate user interfaces (UIs) for 360 VR navigation within the limitations of fixed assets. To tackle this gap, we …

Remembering The City: An Augmented Reality Reconstruction Of Memory, Power, And Identity In Ho Chi Minh City Through Cartography & Architecture, Thuy Dinh

Senior Independent Study Theses

Cartography and architecture are official channels that facilitate remembrance in Ho Chi Minh City. Maps and buildings serve as sites for actors of memory to manipulate the city's narratives and shape its collective identity. Power enables the production of space and knowledge through sites of memory. The ruling regimes of Ho Chi Minh City have leveraged control over the natural environment and the local population to create new forms of materials that propagate their ideologies and ideals for the city. Alterations to the natural and built environments in the city legitimize the authorities' official narratives for its history and future …

Automatic Gaze Classification For Aviators: Using Multi-Task Convolutional Networks As A Proxy For Flight Instructor Observation, Justin Wilson, Sandro Scielzo, Sukumaran Nair, Eric C. Larson

International Journal of Aviation, Aeronautics, and Aerospace

In this work, we investigate how flight instructors observe aviator scan patterns and assign quality to an aviator's gaze. We first establish the reliability of instructors to assign similar quality to an aviator's scan patterns, and then investigate methods to automate this quality using machine learning. In particular, we focus on the classification of gaze for aviators in a mixed-reality flight simulation. We create and evaluate two machine learning models for classifying gaze quality of aviators: a task-agnostic model and a multi-task model. Both models use deep convolutional neural networks to classify the quality of pilot gaze patterns for 40 …

Virtual Reality Environment Recreation, Ryan Douglas

Williams Honors College, Honors Research Projects

This project will consist of a virtual reality based program that is capable of showing the user both the modern day state of a site of historic or archaeological significance, along with a recreation of what said site or area may have looked like in the past, primarily during the time that gave the site its historical significance. The virtual reality program itself is to be run on modern day Windows hardware and used with the VIVE virtual reality head-mounted display and controllers. Alongside the completed program, the creation of the environments themselves will be documented, resulting in an organized …

Feature Extraction And Description For Retinal Fundus Image Registration, Ramli Roziana

Student Works (2020-2029)

Retinal fundus image registration (RIR) is performed to align two or more fundus images. A general framework of a feature-based RIR technique comprises of preprocessing, feature extraction, feature descriptor, matching and estimating geometrical transformation. The RIR is mainly performed for super-resolution, image mosaicking and longitudinal study applications to assist diagnosis and monitoring retinal diseases. Registering image pair from these applications involve a combination of challenges such as overlapping area and rotation between images. The challenges of the overlapping area and rotation can be addressed at feature extraction and feature descriptor stages of the feature-based RIR technique, respectively. To address the …

A Saliency-Driven Video Magnifier For People With Low Vision, Ali Selman Aydin, Shirin Feiz, Iv Ramakrishnan, Vikas Ashok

Computer Science Faculty Publications

Consuming video content poses significant challenges for many screen magnifier users, which is the “go to” assistive technology for people with low vision. While screen magnifier software could be used to achieve a zoom factor that would make the content of the video visible to low-vision users, it is oftentimes a major challenge for these users to navigate through videos. Towards making videos more accessible for low-vision users, we have developed the SViM video magnifier system [6]. Specifically, SViM consists of three different magnifier interfaces with easy-to-use means of interactions. All three interfaces are driven by visual saliency as a …

Towards Making Videos Accessible For Low Vision Screen Magnifier Users, Ali Selman Aydin, Shirin Feiz, Vikas Ashok, Iv Ramakrishnan

Computer Science Faculty Publications

People with low vision who use screen magnifiers to interact with computing devices find it very challenging to interact with dynamically changing digital content such as videos, since they do not have the luxury of time to manually move, i.e., pan the magnifier lens to different regions of interest (ROIs) or zoom into these ROIs before the content changes across frames.

In this paper, we present SViM, a first of its kind screen-magnifier interface for such users that leverages advances in computer vision, particularly video saliency models, to identify salient ROIs in videos. SViM's interface allows users to zoom in/out …

Accessibility Of Deepfakes, Andrew L. Collings

Cybersecurity Undergraduate Research Showcase

The danger posed by falsified media, commonly referred to as deepfakes, has been well researched and documented. The software Faceswap to was used to swap the faces of two politician (Joe Biden and Donald Trump). The testing was performed using an affordable consumer GPU (an AMD Radeon RX 570) over 100,000 iterations. The process and results for the two attempts with the best results (and largest differences) were recorded. The result was ultimately unconvincing, while the software was able to recreate the facial structure the lighting and skin tone did not blend at all.

Stylized 2d Fabrication Of Non-Photorealistic Images, Athina Panotopoulou

Dartmouth College Ph.D Dissertations

A current trend in computer graphics is the use of programmable tools that allow non-experts to engage in the design of physical prototypes. Within fabrication, one area of research focuses on non-photorealistic images which are stylized to depict a particular aesthetic quality or convey key information. In cases where authenticity is demanded or the images need to be manipulated, fabrication is necessary. Non-photorealistic image fabrication involves two challenges: identifying and abstracting key information during design and considering material restrictions during fabrication. This thesis showcases two examples for fabricating new types of non-photorealistic images, the first involving watercolors, and the second …

Neighbourhood Structure Preserving Cross-Modal Embedding For Video Hyperlinking, Yanbin Hao, Chong-Wah Ngo, Benoit Huet

Research Collection School Of Computing and Information Systems

Video hyperlinking is a task aiming to enhance the accessibility of large archives, by establishing links between fragments of videos. The links model the aboutness between fragments for efficient traversal of video content. This paper addresses the problem of link construction from the perspective of cross-modal embedding. To this end, a generalized multi-modal auto-encoder is proposed.& x00A0;The encoder learns two embeddings from visual and speech modalities, respectively, whereas each of the embeddings performs self-modal and cross-modal translation of modalities. Furthermore, to preserve the neighbourhood structure of fragments, which is important for video hyperlinking, the auto-encoder is devised to model data …

An Intra-Severity Classification And Adaptation Technique To Improve Dysarthric Speech Recognition Accuracy, Al-Qatab Bassam Ali Qasem

Student Works (2020-2029)

Dysarthria is a motor speech impairment at the neurological and/or muscular levels that caused difficulty in pronouncing words clearly. Automatic speech recognition (ASR) system is increasingly applied as assistive technology to aid an individual with physical disability particularly the speech impaired community such as dysarthria speakers. However, the development of an effective ASR system is hindered by the data sparsity, either in the coverage of the language or the size of the existing speech databases. The speaker adaptation (SA) technique is one of the solutions to overcome the data sparsity issue of ASR for dysarthric speakers. Our proposed method introduces …

Web Content Management System And Accessibility Awareness: A Comparative Study Of Novice Users And Accessibility Outcomes, Fatima Artiba Diaz

Theses: Doctorates and Masters

Since its creation, the Web has progressively developed and become a vital source of information in every domain and for almost all people. It is crucial to guarantee that the information contained on the Web is available for everyone, especially for people with special needs. Removing accessibility barriers is fundamentally based on tools, skills and support of all contributors, particularly the content creators, to ensure information is navigable and usable in the context of the end users experience. Web Content Management Systems play a significant role in structuring, storing and provision content to the Web and have evolved to address …

Vireo @ Video Browser Showdown 2020, Phuong Anh Nguyen, Jiaxin Wu, Chong-Wah Ngo, Danny Francis, Benoit Huet

Research Collection School Of Computing and Information Systems

In this paper, we present the features implemented in the 4th version of the VIREO Video Search System (VIREO-VSS). In this version, we propose a sketch-based retrieval model, which allows the user to specify a video scene with objects and their basic properties, including color, size, and location. We further utilize the temporal relation between video frames to strengthen this retrieval model. For text-based retrieval module, we supply speech and on-screen text for free-text search and upgrade the concept bank for concept search. The search interface is also re-designed targeting the novice user. With the introduced system, we expect that …

Deepdrawing: A Deep Learning Approach To Graph Drawing, Yong Wang, Zhihua Jin, Qianwen Wang, Weiwei Cui, Tengfei Ma, Huamin Qu

Research Collection School Of Computing and Information Systems

Node-link diagrams are widely used to facilitate network explorations. However, when using a graph drawing technique to visualize networks, users often need to tune different algorithm-specific parameters iteratively by comparing the corresponding drawing results in order to achieve a desired visual effect. This trial and error process is often tedious and time-consuming, especially for non-expert users. Inspired by the powerful data modelling and prediction capabilities of deep learning techniques, we explore the possibility of applying deep learning techniques to graph drawing. Specifically, we propose using a graph-LSTM-based approach to directly map network structures to graph drawings. Given a set of …

Planningvis: A Visual Analytics Approach To Production Planning In Smart Factories, Dong Sun, Renfei Huang, Yuanzhe Chen, Yong Wang, Jia Zeng, Mingxuan Yuan, Ting-Chuen Pong, Huamin Qu

Research Collection School Of Computing and Information Systems

Production planning in the manufacturing industry is crucial for fully utilizing factory resources (e.g., machines, raw materials and workers) and reducing costs. With the advent of industry 4.0, plenty of data recording the status of factory resources have been collected and further involved in production planning, which brings an unprecedented opportunity to understand, evaluate and adjust complex production plans through a data-driven approach. However, developing a systematic analytics approach for production planning is challenging due to the large volume of production data, the complex dependency between products, and unexpected changes in the market and the plant. Previous studies only provide …

Towards Automated Infographic Design: Deep Learning-Based Auto-Extraction Of Extensible Timeline, Zhutian Chen, Yun Wang, Qianwen Wang, Yong Wang, Huamin Qu

Research Collection School Of Computing and Information Systems

Designers need to consider not only perceptual effectiveness but also visual styles when creating an infographic. This process can be difficult and time consuming for professional designers, not to mention non-expert users, leading to the demand for automated infographics design. As a first step, we focus on timeline infographics, which have been widely used for centuries. We contribute an end-to-end approach that automatically extracts an extensible timeline template from a bitmap image. Our approach adopts a deconstruction and reconstruction paradigm. At the deconstruction stage, we propose a multi-task deep neural network that simultaneously parses two kinds of information from a …

A Desire Fulfillment Theory Of Digital Game Enjoyment, Owen M. Schaffer

College of Computing and Digital Media Dissertations

Empirical research on what makes digital games enjoyable is critical for practitioners who want to design for enjoyment, including for Game Design, Gamification, and Serious Games. But existing theories of what leads to digital game enjoyment have been incomplete or lacking in empirical support showing their impact on enjoyment.

Desire Fulfillment Theory is proposed as a new theory of what leads to digital game enjoyment and tested through research with people who have recently played a digital game. This theory builds on three established theories: Expectancy Disconfirmation Theory, Theory of Basic Human Desires, and Flow Theory. These three theories are …

Image Classification Using Fuzzy Fca, Niruktha Roy Gotoor

School of Computing: Dissertations, Theses, and Student Research

Formal concept analysis (FCA) is a mathematical theory based on lattice and order theory used for data analysis and knowledge representation. It has been used in various domains such as data mining, machine learning, semantic web, Sciences, for the purpose of data analysis and Ontology over the last few decades. Various extensions of FCA are being researched to expand it's scope over more departments. In this thesis,we review the theory of Formal Concept Analysis (FCA) and its extension Fuzzy FCA. Many studies to use FCA in data mining and text learning have been pursued. We extend these studies to include …

Improved Generalisation Bounds For Deep Learning Through L∞ Covering Numbers, Antoine Ledent, Yunwen Lei, Marius Kloft

Research Collection School Of Computing and Information Systems

Using proof techniques involving L∞ covering numbers, we show generalisation error bounds for deep learning with two main improvements over the state of the art. First, our bounds have no explicit dependence on the number of classes except for logarithmic factors. This holds even when formulating the bounds in terms of the L 2 norm of the weight matrices, while previous bounds exhibit at least a square-root dependence on the number of classes in this case. Second, we adapt the Rademacher analysis of DNNs to incorporate weight sharing—a task of fundamental theoretical importance which was previously attempted only under very …

Efficient Meta Learning Via Minibatch Proximal Update, Pan Zhou, Xiao-Tong Yuan, Huan Xu, Shuicheng Yan, Jiashi Feng

Research Collection School Of Computing and Information Systems

We address the problem of meta-learning which learns a prior over hypothesis from a sample of meta-training tasks for fast adaptation on meta-testing tasks. A particularly simple yet successful paradigm for this research is model-agnostic meta-learning (MAML). Implementation and analysis of MAML, however, can be tricky; first-order approximation is usually adopted to avoid directly computing Hessian matrix but as a result the convergence and generalization guarantees remain largely mysterious for MAML. To remedy this deficiency, in this paper we propose a minibatch proximal update based meta-learning approach for learning to efficient hypothesis transfer. The principle is to learn a prior …

Improving Medication Information Presentation Through Interactive Visualization In Mobile Apps: Human Factors Design, Don Roosan, Yan Li, Anandi Law, Huy Truong, Mazharul Karim, Jay Chok, Moom Roosan

Pharmacy Faculty Articles and Research

Background: Despite the detailed patient package inserts (PPIs) with prescription drugs that communicate crucial information about safety, there is a critical gap between patient understanding and the knowledge presented. As a result, patients may suffer from adverse events. We propose using human factors design methodologies such as hierarchical task analysis (HTA) and interactive visualization to bridge this gap. We hypothesize that an innovative mobile app employing human factors design with an interactive visualization can deliver PPI information aligned with patients’ information processing heuristics. Such an app may help patients gain an improved overall knowledge of medications.

Objective: The …