Computer Sciences Commons^™

Coding For The Public Good: Front-End Website Design And Development, Devorah Kletenik

Open Educational Resources

This activity helps student design and develop a front-end of a website, from wireframes through HTML/CSS/Javascript. It includes design questions for students, including the invocation of Ben Schneiderman's eight golden rules for interface design.

Note: this activity assumes prior knowledge of web development. Since this activity is designed for an HCI course, with a focus on interface design, students are not expected to create a back-end for it. This activity can obviously be modified for a full-stack experience.

Accessibility Evaluation, Devorah Kletenik

Open Educational Resources

This activity guides students through the evaluation of a website that they have created to see if it is accessible for users with disabilities. Students will simulate a number of different disabilities (e.g. visual impairments, color blindness, auditory impairments, motor impairments) to see if their website is accessible; they will also use automated W3 and WAVE tools to evaluate their sites. Students will consider the needs of users with disabilities by creating a persona and scenario of a user with disabilities interacting with their site. Finally, students will write up recommendations to change their site and implement the changes.

Interactions Between Humans, Virtual Agent Characters And Virtual Avatars, Tamara Griffith

Electronic Theses and Dissertations, 2020-2023

Simulations allow people to experience events as if they were happening in the real world in a way that is safer and less expensive than live training. Despite improvements in realism in simulated environments, one area that still presents a challenge is interpersonal interactions. The subtleties of what makes an interaction rich are difficult to define. We may never fully understand the complexity of human interchanges, however there is value in building on existing research into how individuals react to virtual characters to inform future investments. Virtual characters can either be automated through computational processes, referred to as agents, or …

Toward Efficient Automation Of Interpretable Machine Learning Boosting, Nathan Neuhaus

All Master's Theses

Developing efficient automated methods for Interpretable Machine Learning (IML) is an important and long-term goal in the field of Artificial Intelligence. Currently the Machine Learning landscape is dominated by Neural Networks (NNs) and Support Vector Machines (SVMs), models which are often highly accurate. Despite high accuracy, such models are essentially “black boxes” and therefore are too risky for situations like healthcare where real lives are at stake. In such situations, so called “glass-box” models, such as Decision Trees (DTs), Bayesian Networks (BNs), and Logic Relational (LR) models are often preferred, however can succumb to accuracy limitations. Unfortunately, having to choose …

Remembering The City: An Augmented Reality Reconstruction Of Memory, Power, And Identity In Ho Chi Minh City Through Cartography & Architecture, Thuy Dinh

Senior Independent Study Theses

Cartography and architecture are official channels that facilitate remembrance in Ho Chi Minh City. Maps and buildings serve as sites for actors of memory to manipulate the city's narratives and shape its collective identity. Power enables the production of space and knowledge through sites of memory. The ruling regimes of Ho Chi Minh City have leveraged control over the natural environment and the local population to create new forms of materials that propagate their ideologies and ideals for the city. Alterations to the natural and built environments in the city legitimize the authorities' official narratives for its history and future …

Where Did The Time Go?, John C. Viaud, Bilal Abdulmajid, Vitali Surmach, Jia Yanxia

Capstone Showcase

Study shows that most people spend a full quarter of their active hours on their mobile device which can take a serious toll on our productivity as well as our mental and physical well-being. We created an Android app that is able to track and visualize phone usage patterns to help user establish awareness of how much and in what ways they use their Android devices. Unlike currently existing phone time management apps, such as Apple’s ScreenTime and Google’s new Digital Wellbeing, our app provides functionalities to encourage off-phone time and personal goal management.

Android Game, Ryan Weston

Williams Honors College, Honors Research Projects

The purpose of this project was to create an endless runner game for Android coded in Java and XML and developed in Android Studio. In the game, the player controls a frog that jumps from lily pad to lily pad to avoid logs moving toward the player. The player must also maneuver the lily pads as they can randomly disappear. There are three difficulties in the game that vary the disappearance rate of lily pads as well as the frequency and acceleration rate of the log obstacles. The game also has a scoring system and saves the high score locally …

Virtual Reality Environment Recreation, Ryan Douglas

Williams Honors College, Honors Research Projects

This project will consist of a virtual reality based program that is capable of showing the user both the modern day state of a site of historic or archaeological significance, along with a recreation of what said site or area may have looked like in the past, primarily during the time that gave the site its historical significance. The virtual reality program itself is to be run on modern day Windows hardware and used with the VIVE virtual reality head-mounted display and controllers. Alongside the completed program, the creation of the environments themselves will be documented, resulting in an organized …

Stylized 2d Fabrication Of Non-Photorealistic Images, Athina Panotopoulou

Dartmouth College Ph.D Dissertations

A current trend in computer graphics is the use of programmable tools that allow non-experts to engage in the design of physical prototypes. Within fabrication, one area of research focuses on non-photorealistic images which are stylized to depict a particular aesthetic quality or convey key information. In cases where authenticity is demanded or the images need to be manipulated, fabrication is necessary. Non-photorealistic image fabrication involves two challenges: identifying and abstracting key information during design and considering material restrictions during fabrication. This thesis showcases two examples for fabricating new types of non-photorealistic images, the first involving watercolors, and the second …

An Intra-Severity Classification And Adaptation Technique To Improve Dysarthric Speech Recognition Accuracy, Al-Qatab Bassam Ali Qasem

Student Works (2020-2029)

Dysarthria is a motor speech impairment at the neurological and/or muscular levels that caused difficulty in pronouncing words clearly. Automatic speech recognition (ASR) system is increasingly applied as assistive technology to aid an individual with physical disability particularly the speech impaired community such as dysarthria speakers. However, the development of an effective ASR system is hindered by the data sparsity, either in the coverage of the language or the size of the existing speech databases. The speaker adaptation (SA) technique is one of the solutions to overcome the data sparsity issue of ASR for dysarthric speakers. Our proposed method introduces …

V-Slam And Sensor Fusion For Ground Robots, Ejup Hoxha

Dissertations and Theses

In underground, underwater and indoor environments, a robot has to rely solely on its on-board sensors to sense and understand its surroundings. This is the main reason why SLAM gained the popularity it has today. In recent years, we have seen excellent improvement on accuracy of localization using cameras and combinations of different sensors, especially camera-IMU (VIO) fusion. Incorporating more sensors leads to improvement of accuracy,but also robustness of SLAM. However, while testing SLAM in our ground robots, we have seen a decrease in performance quality when using the same algorithms on flying vehicles.We have an additional sensor for ground …

Feature Extraction And Description For Retinal Fundus Image Registration, Ramli Roziana

Student Works (2020-2029)

Retinal fundus image registration (RIR) is performed to align two or more fundus images. A general framework of a feature-based RIR technique comprises of preprocessing, feature extraction, feature descriptor, matching and estimating geometrical transformation. The RIR is mainly performed for super-resolution, image mosaicking and longitudinal study applications to assist diagnosis and monitoring retinal diseases. Registering image pair from these applications involve a combination of challenges such as overlapping area and rotation between images. The challenges of the overlapping area and rotation can be addressed at feature extraction and feature descriptor stages of the feature-based RIR technique, respectively. To address the …

Automatic Gaze Classification For Aviators: Using Multi-Task Convolutional Networks As A Proxy For Flight Instructor Observation, Justin Wilson, Sandro Scielzo, Sukumaran Nair, Eric C. Larson

International Journal of Aviation, Aeronautics, and Aerospace

In this work, we investigate how flight instructors observe aviator scan patterns and assign quality to an aviator's gaze. We first establish the reliability of instructors to assign similar quality to an aviator's scan patterns, and then investigate methods to automate this quality using machine learning. In particular, we focus on the classification of gaze for aviators in a mixed-reality flight simulation. We create and evaluate two machine learning models for classifying gaze quality of aviators: a task-agnostic model and a multi-task model. Both models use deep convolutional neural networks to classify the quality of pilot gaze patterns for 40 …

A Saliency-Driven Video Magnifier For People With Low Vision, Ali Selman Aydin, Shirin Feiz, Iv Ramakrishnan, Vikas Ashok

Computer Science Faculty Publications

Consuming video content poses significant challenges for many screen magnifier users, which is the “go to” assistive technology for people with low vision. While screen magnifier software could be used to achieve a zoom factor that would make the content of the video visible to low-vision users, it is oftentimes a major challenge for these users to navigate through videos. Towards making videos more accessible for low-vision users, we have developed the SViM video magnifier system [6]. Specifically, SViM consists of three different magnifier interfaces with easy-to-use means of interactions. All three interfaces are driven by visual saliency as a …

Repurposing Visual Input Modalities For Blind Users: A Case Study Of Word Processors, Hae-Na Lee, Vikas Ashok, I.V. Ramakrishnan

Computer Science Faculty Publications

Visual 'point-and-click' interaction artifacts such as mouse and touchpad are tangible input modalities, which are essential for sighted users to conveniently interact with computer applications. In contrast, blind users are unable to leverage these visual input modalities and are thus limited while interacting with computers using a sequentially narrating screen-reader assistive technology that is coupled to keyboards. As a consequence, blind users generally require significantly more time and effort to do even simple application tasks (e.g., applying a style to text in a word processor) using only keyboard, compared to their sighted peers who can effortlessly accomplish the same tasks …

Towards Making Videos Accessible For Low Vision Screen Magnifier Users, Ali Selman Aydin, Shirin Feiz, Vikas Ashok, Iv Ramakrishnan

Computer Science Faculty Publications

People with low vision who use screen magnifiers to interact with computing devices find it very challenging to interact with dynamically changing digital content such as videos, since they do not have the luxury of time to manually move, i.e., pan the magnifier lens to different regions of interest (ROIs) or zoom into these ROIs before the content changes across frames.

In this paper, we present SViM, a first of its kind screen-magnifier interface for such users that leverages advances in computer vision, particularly video saliency models, to identify salient ROIs in videos. SViM's interface allows users to zoom in/out …

Rotate-And-Press: A Non-Visual Alternative To Point-And-Click, Hae-Na Lee, Vikas Ashok, I. V. Ramakrishnan

Computer Science Faculty Publications

Most computer applications manifest visually rich and dense graphical user interfaces (GUIs) that are primarily tailored for an easy-and-efficient sighted interaction using a combination of two default input modalities, namely the keyboard and the mouse/touchpad. However, blind screen-reader users predominantly rely only on keyboard, and therefore struggle to interact with these applications, since it is both arduous and tedious to perform the visual 'point-and-click' tasks such as accessing the various application commands/features using just keyboard shortcuts supported by screen readers.

In this paper, we investigate the suitability of a 'rotate-and-press' input modality as an effective non-visual substitute for the visual …

Sail: Saliency-Driven Injection Of Aria Landmarks, Ali Selman Aydin, Shirin Feiz, Vikas Ashok, Iv Ramakrishnan

Computer Science Faculty Publications

Navigating webpages with screen readers is a challenge even with recent improvements in screen reader technologies and the increased adoption of web standards for accessibility, namely ARIA. ARIA landmarks, an important aspect of ARIA, lets screen reader users access different sections of the webpage quickly, by enabling them to skip over blocks of irrelevant or redundant content. However, these landmarks are sporadically and inconsistently used by web developers, and in many cases, even absent in numerous web pages. Therefore, we propose SaIL, a scalable approach that automatically detects the important sections of a web page, and then injects ARIA landmarks …

Web Content Management System And Accessibility Awareness: A Comparative Study Of Novice Users And Accessibility Outcomes, Fatima Artiba Diaz

Theses: Doctorates and Masters

Since its creation, the Web has progressively developed and become a vital source of information in every domain and for almost all people. It is crucial to guarantee that the information contained on the Web is available for everyone, especially for people with special needs. Removing accessibility barriers is fundamentally based on tools, skills and support of all contributors, particularly the content creators, to ensure information is navigable and usable in the context of the end users experience. Web Content Management Systems play a significant role in structuring, storing and provision content to the Web and have evolved to address …

Deepdrawing: A Deep Learning Approach To Graph Drawing, Yong Wang, Zhihua Jin, Qianwen Wang, Weiwei Cui, Tengfei Ma, Huamin Qu

Research Collection School Of Computing and Information Systems

Node-link diagrams are widely used to facilitate network explorations. However, when using a graph drawing technique to visualize networks, users often need to tune different algorithm-specific parameters iteratively by comparing the corresponding drawing results in order to achieve a desired visual effect. This trial and error process is often tedious and time-consuming, especially for non-expert users. Inspired by the powerful data modelling and prediction capabilities of deep learning techniques, we explore the possibility of applying deep learning techniques to graph drawing. Specifically, we propose using a graph-LSTM-based approach to directly map network structures to graph drawings. Given a set of …

Neighbourhood Structure Preserving Cross-Modal Embedding For Video Hyperlinking, Yanbin Hao, Chong-Wah Ngo, Benoit Huet

Research Collection School Of Computing and Information Systems

Video hyperlinking is a task aiming to enhance the accessibility of large archives, by establishing links between fragments of videos. The links model the aboutness between fragments for efficient traversal of video content. This paper addresses the problem of link construction from the perspective of cross-modal embedding. To this end, a generalized multi-modal auto-encoder is proposed.& x00A0;The encoder learns two embeddings from visual and speech modalities, respectively, whereas each of the embeddings performs self-modal and cross-modal translation of modalities. Furthermore, to preserve the neighbourhood structure of fragments, which is important for video hyperlinking, the auto-encoder is devised to model data …

Vireo @ Video Browser Showdown 2020, Phuong Anh Nguyen, Jiaxin Wu, Chong-Wah Ngo, Danny Francis, Benoit Huet

Research Collection School Of Computing and Information Systems

In this paper, we present the features implemented in the 4th version of the VIREO Video Search System (VIREO-VSS). In this version, we propose a sketch-based retrieval model, which allows the user to specify a video scene with objects and their basic properties, including color, size, and location. We further utilize the temporal relation between video frames to strengthen this retrieval model. For text-based retrieval module, we supply speech and on-screen text for free-text search and upgrade the concept bank for concept search. The search interface is also re-designed targeting the novice user. With the introduced system, we expect that …

Planningvis: A Visual Analytics Approach To Production Planning In Smart Factories, Dong Sun, Renfei Huang, Yuanzhe Chen, Yong Wang, Jia Zeng, Mingxuan Yuan, Ting-Chuen Pong, Huamin Qu

Research Collection School Of Computing and Information Systems

Production planning in the manufacturing industry is crucial for fully utilizing factory resources (e.g., machines, raw materials and workers) and reducing costs. With the advent of industry 4.0, plenty of data recording the status of factory resources have been collected and further involved in production planning, which brings an unprecedented opportunity to understand, evaluate and adjust complex production plans through a data-driven approach. However, developing a systematic analytics approach for production planning is challenging due to the large volume of production data, the complex dependency between products, and unexpected changes in the market and the plant. Previous studies only provide …

Towards Automated Infographic Design: Deep Learning-Based Auto-Extraction Of Extensible Timeline, Zhutian Chen, Yun Wang, Qianwen Wang, Yong Wang, Huamin Qu

Research Collection School Of Computing and Information Systems

Designers need to consider not only perceptual effectiveness but also visual styles when creating an infographic. This process can be difficult and time consuming for professional designers, not to mention non-expert users, leading to the demand for automated infographics design. As a first step, we focus on timeline infographics, which have been widely used for centuries. We contribute an end-to-end approach that automatically extracts an extensible timeline template from a bitmap image. Our approach adopts a deconstruction and reconstruction paradigm. At the deconstruction stage, we propose a multi-task deep neural network that simultaneously parses two kinds of information from a …

Intelligent Cinematic Camera Control For Real-Time Graphics Applications, Ian Harris Meeder

Master's Theses

E-sports is currently estimated to be a billion dollar industry which is only growing in size from year to year. However the cinematography of spectated games leaves much to be desired. In most cases, the spectator either gets to control their own freely-moving camera or they get to see the view that a specific player sees. This thesis presents a system for the generation of cinematically-pleasing views for spectating real-time graphics applications. A custom real-time engine has been built to demonstrate the effect of this system on several different game modes with varying visual cinematic constraints, such as the rule …

A Desire Fulfillment Theory Of Digital Game Enjoyment, Owen M. Schaffer

College of Computing and Digital Media Dissertations

Empirical research on what makes digital games enjoyable is critical for practitioners who want to design for enjoyment, including for Game Design, Gamification, and Serious Games. But existing theories of what leads to digital game enjoyment have been incomplete or lacking in empirical support showing their impact on enjoyment.

Desire Fulfillment Theory is proposed as a new theory of what leads to digital game enjoyment and tested through research with people who have recently played a digital game. This theory builds on three established theories: Expectancy Disconfirmation Theory, Theory of Basic Human Desires, and Flow Theory. These three theories are …

Image Classification Using Fuzzy Fca, Niruktha Roy Gotoor

School of Computing: Dissertations, Theses, and Student Research

Formal concept analysis (FCA) is a mathematical theory based on lattice and order theory used for data analysis and knowledge representation. It has been used in various domains such as data mining, machine learning, semantic web, Sciences, for the purpose of data analysis and Ontology over the last few decades. Various extensions of FCA are being researched to expand it's scope over more departments. In this thesis,we review the theory of Formal Concept Analysis (FCA) and its extension Fuzzy FCA. Many studies to use FCA in data mining and text learning have been pursued. We extend these studies to include …

Efficient Meta Learning Via Minibatch Proximal Update, Pan Zhou, Xiao-Tong Yuan, Huan Xu, Shuicheng Yan, Jiashi Feng

Research Collection School Of Computing and Information Systems

We address the problem of meta-learning which learns a prior over hypothesis from a sample of meta-training tasks for fast adaptation on meta-testing tasks. A particularly simple yet successful paradigm for this research is model-agnostic meta-learning (MAML). Implementation and analysis of MAML, however, can be tricky; first-order approximation is usually adopted to avoid directly computing Hessian matrix but as a result the convergence and generalization guarantees remain largely mysterious for MAML. To remedy this deficiency, in this paper we propose a minibatch proximal update based meta-learning approach for learning to efficient hypothesis transfer. The principle is to learn a prior …

Improved Generalisation Bounds For Deep Learning Through L∞ Covering Numbers, Antoine Ledent, Yunwen Lei, Marius Kloft

Research Collection School Of Computing and Information Systems

Using proof techniques involving L∞ covering numbers, we show generalisation error bounds for deep learning with two main improvements over the state of the art. First, our bounds have no explicit dependence on the number of classes except for logarithmic factors. This holds even when formulating the bounds in terms of the L 2 norm of the weight matrices, while previous bounds exhibit at least a square-root dependence on the number of classes in this case. Second, we adapt the Rademacher analysis of DNNs to incorporate weight sharing—a task of fundamental theoretical importance which was previously attempted only under very …

Improving Medication Information Presentation Through Interactive Visualization In Mobile Apps: Human Factors Design, Don Roosan, Yan Li, Anandi Law, Huy Truong, Mazharul Karim, Jay Chok, Moom Roosan

Pharmacy Faculty Articles and Research

Background: Despite the detailed patient package inserts (PPIs) with prescription drugs that communicate crucial information about safety, there is a critical gap between patient understanding and the knowledge presented. As a result, patients may suffer from adverse events. We propose using human factors design methodologies such as hierarchical task analysis (HTA) and interactive visualization to bridge this gap. We hypothesize that an innovative mobile app employing human factors design with an interactive visualization can deliver PPI information aligned with patients’ information processing heuristics. Such an app may help patients gain an improved overall knowledge of medications.

Objective: The …