Graphics and Human Computer Interfaces Commons^™

Sail: Saliency-Driven Injection Of Aria Landmarks, Ali Selman Aydin, Shirin Feiz, Vikas Ashok, Iv Ramakrishnan

Computer Science Faculty Publications

Navigating webpages with screen readers is a challenge even with recent improvements in screen reader technologies and the increased adoption of web standards for accessibility, namely ARIA. ARIA landmarks, an important aspect of ARIA, lets screen reader users access different sections of the webpage quickly, by enabling them to skip over blocks of irrelevant or redundant content. However, these landmarks are sporadically and inconsistently used by web developers, and in many cases, even absent in numerous web pages. Therefore, we propose SaIL, a scalable approach that automatically detects the important sections of a web page, and then injects ARIA landmarks …

Rotate-And-Press: A Non-Visual Alternative To Point-And-Click, Hae-Na Lee, Vikas Ashok, I. V. Ramakrishnan

Computer Science Faculty Publications

Most computer applications manifest visually rich and dense graphical user interfaces (GUIs) that are primarily tailored for an easy-and-efficient sighted interaction using a combination of two default input modalities, namely the keyboard and the mouse/touchpad. However, blind screen-reader users predominantly rely only on keyboard, and therefore struggle to interact with these applications, since it is both arduous and tedious to perform the visual 'point-and-click' tasks such as accessing the various application commands/features using just keyboard shortcuts supported by screen readers.

In this paper, we investigate the suitability of a 'rotate-and-press' input modality as an effective non-visual substitute for the visual …

Vireo @ Video Browser Showdown 2020, Phuong Anh Nguyen, Jiaxin Wu, Chong-Wah Ngo, Danny Francis, Benoit Huet

Research Collection School Of Computing and Information Systems

In this paper, we present the features implemented in the 4th version of the VIREO Video Search System (VIREO-VSS). In this version, we propose a sketch-based retrieval model, which allows the user to specify a video scene with objects and their basic properties, including color, size, and location. We further utilize the temporal relation between video frames to strengthen this retrieval model. For text-based retrieval module, we supply speech and on-screen text for free-text search and upgrade the concept bank for concept search. The search interface is also re-designed targeting the novice user. With the introduced system, we expect that …

Neighbourhood Structure Preserving Cross-Modal Embedding For Video Hyperlinking, Yanbin Hao, Chong-Wah Ngo, Benoit Huet

Research Collection School Of Computing and Information Systems

Video hyperlinking is a task aiming to enhance the accessibility of large archives, by establishing links between fragments of videos. The links model the aboutness between fragments for efficient traversal of video content. This paper addresses the problem of link construction from the perspective of cross-modal embedding. To this end, a generalized multi-modal auto-encoder is proposed.& x00A0;The encoder learns two embeddings from visual and speech modalities, respectively, whereas each of the embeddings performs self-modal and cross-modal translation of modalities. Furthermore, to preserve the neighbourhood structure of fragments, which is important for video hyperlinking, the auto-encoder is devised to model data …

Deepdrawing: A Deep Learning Approach To Graph Drawing, Yong Wang, Zhihua Jin, Qianwen Wang, Weiwei Cui, Tengfei Ma, Huamin Qu

Research Collection School Of Computing and Information Systems

Node-link diagrams are widely used to facilitate network explorations. However, when using a graph drawing technique to visualize networks, users often need to tune different algorithm-specific parameters iteratively by comparing the corresponding drawing results in order to achieve a desired visual effect. This trial and error process is often tedious and time-consuming, especially for non-expert users. Inspired by the powerful data modelling and prediction capabilities of deep learning techniques, we explore the possibility of applying deep learning techniques to graph drawing. Specifically, we propose using a graph-LSTM-based approach to directly map network structures to graph drawings. Given a set of …

Planningvis: A Visual Analytics Approach To Production Planning In Smart Factories, Dong Sun, Renfei Huang, Yuanzhe Chen, Yong Wang, Jia Zeng, Mingxuan Yuan, Ting-Chuen Pong, Huamin Qu

Research Collection School Of Computing and Information Systems

Production planning in the manufacturing industry is crucial for fully utilizing factory resources (e.g., machines, raw materials and workers) and reducing costs. With the advent of industry 4.0, plenty of data recording the status of factory resources have been collected and further involved in production planning, which brings an unprecedented opportunity to understand, evaluate and adjust complex production plans through a data-driven approach. However, developing a systematic analytics approach for production planning is challenging due to the large volume of production data, the complex dependency between products, and unexpected changes in the market and the plant. Previous studies only provide …

Towards Automated Infographic Design: Deep Learning-Based Auto-Extraction Of Extensible Timeline, Zhutian Chen, Yun Wang, Qianwen Wang, Yong Wang, Huamin Qu

Research Collection School Of Computing and Information Systems

Designers need to consider not only perceptual effectiveness but also visual styles when creating an infographic. This process can be difficult and time consuming for professional designers, not to mention non-expert users, leading to the demand for automated infographics design. As a first step, we focus on timeline infographics, which have been widely used for centuries. We contribute an end-to-end approach that automatically extracts an extensible timeline template from a bitmap image. Our approach adopts a deconstruction and reconstruction paradigm. At the deconstruction stage, we propose a multi-task deep neural network that simultaneously parses two kinds of information from a …

Security Camera Using Raspberry Pi, Tejendra Khatri

Student Academic Conference

Making a security camera using raspberry pi utilizing OpenCV for facial recognition, upper body recognition or full-body recognition

Remembering The City: An Augmented Reality Reconstruction Of Memory, Power, And Identity In Ho Chi Minh City Through Cartography & Architecture, Thuy Dinh

Senior Independent Study Theses

Cartography and architecture are official channels that facilitate remembrance in Ho Chi Minh City. Maps and buildings serve as sites for actors of memory to manipulate the city's narratives and shape its collective identity. Power enables the production of space and knowledge through sites of memory. The ruling regimes of Ho Chi Minh City have leveraged control over the natural environment and the local population to create new forms of materials that propagate their ideologies and ideals for the city. Alterations to the natural and built environments in the city legitimize the authorities' official narratives for its history and future …

“Distance Learning” In The Ninth Century?: Micro-Cluster Analysis Of The Epistolary Network Of Alcuin After 796, William James Mattingly

Theses and Dissertations--History

Scholars of eighth- and ninth-century education have assumed that intellectuals did not write works of Scriptural interpretation until that intellectual had a firm foundation in the seven liberal arts.This ensured that anyone who embarked on work of Scriptural interpretation would have the required knowledge and methods to read and interpret Scripture correctly. The potential for theological error and the transmission of those errors was too great unless the interpreter had the requisite training. This dissertation employs computistical methods, specifically the techniques of social network mapping and cluster analysis, to study closely the correspondence of Alcuin, a late-eighth- and early-ninth-century scholar …

Needfinding, Devorah Kletenik

Open Educational Resources

This activity guides students through the process needfinding to identify areas of need for their creation of a technology for the "public good." Students will conduct contextual inquiry to identify the needs of their target audience.

Personas, Scenarios And Storyboards, Devorah Kletenik

Open Educational Resources

This activity guides students towards the creation of personas, scenarios and storyboards for a product/website that they are creating.

Public Interest Technology: Coding For The Public Good, Devorah Kletenik

Open Educational Resources

These slides are used to guide a discussion with students introducing them to the notion of public interest technology and coding for the public good. The lesson is intended to spark a discussion with students about different sorts of technology and their societal ramifications.

Accessibility: The Whys And The Hows, Devorah Kletenik

Open Educational Resources

This presentation introduces Computer Science students to the notion of accessibility: developing software for people with disabilities. This lesson provides a discussion of why accessibility is important (including the legal, societal and ethical benefits) as well as an overview of different types of impairments (visual, auditory, motor, neurological/cognitive) and how developers can make their software accessible to users with those disabilities. This lesson includes videos and links to readings and tutorials for students.

Coding For The Public Good: Front-End Website Design And Development, Devorah Kletenik

Open Educational Resources

This activity helps student design and develop a front-end of a website, from wireframes through HTML/CSS/Javascript. It includes design questions for students, including the invocation of Ben Schneiderman's eight golden rules for interface design.

Note: this activity assumes prior knowledge of web development. Since this activity is designed for an HCI course, with a focus on interface design, students are not expected to create a back-end for it. This activity can obviously be modified for a full-stack experience.

Accessibility Evaluation, Devorah Kletenik

Open Educational Resources

This activity guides students through the evaluation of a website that they have created to see if it is accessible for users with disabilities. Students will simulate a number of different disabilities (e.g. visual impairments, color blindness, auditory impairments, motor impairments) to see if their website is accessible; they will also use automated W3 and WAVE tools to evaluate their sites. Students will consider the needs of users with disabilities by creating a persona and scenario of a user with disabilities interacting with their site. Finally, students will write up recommendations to change their site and implement the changes.

Web Content Management System And Accessibility Awareness: A Comparative Study Of Novice Users And Accessibility Outcomes, Fatima Artiba Diaz

Theses: Doctorates and Masters

Since its creation, the Web has progressively developed and become a vital source of information in every domain and for almost all people. It is crucial to guarantee that the information contained on the Web is available for everyone, especially for people with special needs. Removing accessibility barriers is fundamentally based on tools, skills and support of all contributors, particularly the content creators, to ensure information is navigable and usable in the context of the end users experience. Web Content Management Systems play a significant role in structuring, storing and provision content to the Web and have evolved to address …

Toward Efficient Automation Of Interpretable Machine Learning Boosting, Nathan Neuhaus

All Master's Theses

Developing efficient automated methods for Interpretable Machine Learning (IML) is an important and long-term goal in the field of Artificial Intelligence. Currently the Machine Learning landscape is dominated by Neural Networks (NNs) and Support Vector Machines (SVMs), models which are often highly accurate. Despite high accuracy, such models are essentially “black boxes” and therefore are too risky for situations like healthcare where real lives are at stake. In such situations, so called “glass-box” models, such as Decision Trees (DTs), Bayesian Networks (BNs), and Logic Relational (LR) models are often preferred, however can succumb to accuracy limitations. Unfortunately, having to choose …

Where Did The Time Go?, John C. Viaud, Bilal Abdulmajid, Vitali Surmach, Jia Yanxia

Capstone Showcase

Study shows that most people spend a full quarter of their active hours on their mobile device which can take a serious toll on our productivity as well as our mental and physical well-being. We created an Android app that is able to track and visualize phone usage patterns to help user establish awareness of how much and in what ways they use their Android devices. Unlike currently existing phone time management apps, such as Apple’s ScreenTime and Google’s new Digital Wellbeing, our app provides functionalities to encourage off-phone time and personal goal management.

Android Game, Ryan Weston

Williams Honors College, Honors Research Projects

The purpose of this project was to create an endless runner game for Android coded in Java and XML and developed in Android Studio. In the game, the player controls a frog that jumps from lily pad to lily pad to avoid logs moving toward the player. The player must also maneuver the lily pads as they can randomly disappear. There are three difficulties in the game that vary the disappearance rate of lily pads as well as the frequency and acceleration rate of the log obstacles. The game also has a scoring system and saves the high score locally …