Graphics and Human Computer Interfaces Commons^™

Uit-Adrone: A Novel Drone Dataset For Traffic Anomaly Detection, Tung Minh Tran, Tu N. Vu, Tam Nguyen, Khang Nguyen

Computer Science Faculty Publications

Anomaly detection plays an increasingly important role in video surveillance and is one of the issues that have attracted various communities, such as computer vision, machine learning, and data mining in recent years. Moreover, drones equipped with cameras have quickly been deployed to a wide range of applications, starting from border security applications to street monitoring systems. However, there is a notable lack of adequate drone-based datasets available to detect unusual events in the urban traffic environment, especially in roundabouts, due to the density of interaction between road users and vehicles. To promote the development of anomalous event detection with …

Evaluating Human Eye Features For Objective Measure Of Working Memory Capacity, Yasasi Abeysinghe, Enkelejda Kasneci (Ed.), Frederick Shic (Ed.), Mohamed Khamis (Ed.)

Computer Science Faculty Publications

Eye tracking measures can provide means to understand the underlying development of human working memory. In this study, we propose to develop machine learning algorithms to find an objective relationship between human eye movements via oculomotor plant and their working memory capacity, which determines subjective cognitive load. Here we evaluate oculomotor plant features extracted from saccadic eye movements, traditional positional gaze metrics, and advanced eye metrics such as ambient/focal coefficient , gaze transition entropy, low/high index of pupillary activity (LHIPA), and real-time index of pupillary activity (RIPA). This paper outlines the proposed approach of evaluating eye movements for obtaining an …

Autodesc: Facilitating Convenient Perusal Of Web Data Items For Blind Users, Yash Prakash, Mohan Sunkara, Hae-Na Lee, Sampath Jayarathna, Vikas Ashok

Computer Science Faculty Publications

Web data items such as shopping products, classifieds, and job listings are indispensable components of most e-commerce websites. The information on the data items are typically distributed over two or more webpages, e.g., a ‘Query-Results’ page showing the summaries of the items, and ‘Details’ pages containing full information about the items. While this organization of data mitigates information overload and visual cluttering for sighted users, it however increases the interaction overhead and effort for blind users, as back-and-forth navigation between webpages using screen reader assistive technology is tedious and cumbersome. Existing usability-enhancing solutions are unable to provide adequate support in …

Disease Recognition In X-Ray Images With Doctor Consultation-Inspired Model, Kim Anh Phung, Thuan Trong Nguyen, Nileshkumar Wangad, Samah Baraheem, Nguyen D. Vo, Khang Nguyen

Computer Science Faculty Publications

The application of chest X-ray imaging for early disease screening is attracting interest from the computer vision and deep learning community. To date, various deep learning models have been applied in X-ray image analysis. However, models perform inconsistently depending on the dataset. In this paper, we consider each individual model as a medical doctor. We then propose a doctor consultation-inspired method that fuses multiple models. In particular, we consider both early and late fusion mechanisms for consultation. The early fusion mechanism combines the deep learned features from multiple models, whereas the late fusion method combines the confidence scores of all …

Pervasive Healthcare Internet Of Things: A Survey, Kim Anh Phung, Cemil Kirbas, Leyla Dereci, Tam Van Nguyen

Computer Science Faculty Publications

Thanks to the proliferation of the Internet of Things (IoT), pervasive healthcare is gaining popularity day by day as it offers health support to patients irrespective of their location. In emergency medical situations, medical aid can be sent quickly. Though not yet standardized, this research direction, healthcare Internet of Things (H-IoT), attracts the attention of the research community, both academia and industry. In this article, we conduct a comprehensive survey of pervasive computing H-IoT. We would like to visit the wide range of applications. We provide a broad vision of key components, their roles, and connections in the big picture. …

Few-Shot Object Detection Via Baby Learning, Anh-Khoa Nguyen Vu, Nhat-Duy Nguyen, Khanh-Duy Nguyen, Vinh-Tiep Nguyen, Thanh Duc Ngo, Thanh-Toan Do, Tam Nguyen

Computer Science Faculty Publications

Few-shot learning is proposed to overcome the problem of scarce training data in novel classes. Recently, few-shot learning has been well adopted in various computer vision tasks such as object recognition and object detection. However, the state-of-the-art (SOTA) methods have less attention to effectively reuse the information from previous stages. In this paper, we propose a new framework of few-shot learning for object detection. In particular, we adopt Baby Learning mechanism along with the multiple receptive fields to effectively utilize the former knowledge in novel domain. The propoed framework imitates the learning process of a baby through visual cues. The …

Multi-User Eye-Tracking, Bhanuka Mahanama

Computer Science Faculty Publications

The human gaze characteristics provide informative cues on human behavior during various activities. Using traditional eye trackers, assessing gaze characteristics in the wild requires a dedicated device per participant and therefore is not feasible for large-scale experiments. In this study, we propose a commodity hardware-based multi-user eye-tracking system. We leverage the recent advancements in Deep Neural Networks and large-scale datasets for implementing our system. Our preliminary studies provide promising results for multi-user eye-tracking on commodity hardware, providing a cost-effective solution for large-scale studies.

Toward A Real-Time Index Of Pupillary Activity As An Indicator Of Cognitive Load, Gavindya Jayawardena, Yasith Jayawardana, Sampath Jayarathna, Jonas Högström, Thomas Papa, Deepak Akkil, Andrew T. Duchowski, Vsevolod Peysakhovich, Izabela Krejtz, Nina Gehrer, Krzysztof Krejtz

Computer Science Faculty Publications

The Low/High Index of Pupillary Activity (LHIPA), an eye-tracked measure of pupil diameter oscillation, is redesigned and implemented to function in real-time. The novel Real-time IPA (RIPA) is shown to discriminate cognitive load in re-streamed data from earlier experiments. Rationale for the RIPA is tied to the functioning of the human autonomic nervous system yielding a hybrid measure based on the ratio of Low/High frequencies of pupil oscillation. The paper's contribution is drawn from provision of documentation of the calculation of the RIPA. As with the LHIPA, it is possible for researchers to apply this metric to their own experiments …

Vietnamese Document Analysis: Dataset, Method And Benchmark Suite, Khang Nguyen, An Nguyen, Nguyen D. Vo, Tam Nguyen

Computer Science Faculty Publications

Document image understanding is increasingly useful since the number of digital documents is increasing day-by-day and the need for automation is increasing. Object detection plays a significant role in detecting vital objects and layouts in document images and contributes to providing a clearer understanding of the documents. Nonetheless, previous research mainly focuses on English document images, and studies on Vietnamese document images are limited. In this study, we extensively benchmark state-of-the-art object detectors and analyze the performance of each method on Vietnamese document images. Moreover, we also investigate the effectiveness of four different loss functions on the experimental object detection …

Eye Movement And Pupil Measures: A Review, Bhanuka Mahanama, Yasith Jayawardana, Sundararaman Rengarajan, Gavindya Jayawardena, Leanne Chukoskie, Joseph Snider, Sampath Jayarathna

Computer Science Faculty Publications

Our subjective visual experiences involve complex interaction between our eyes, our brain, and the surrounding world. It gives us the sense of sight, color, stereopsis, distance, pattern recognition, motor coordination, and more. The increasing ubiquity of gaze-aware technology brings with it the ability to track gaze and pupil measures with varying degrees of fidelity. With this in mind, a review that considers the various gaze measures becomes increasingly relevant, especially considering our ability to make sense of these signals given different spatio-temporal sampling capacities. In this paper, we selectively review prior work on eye movements and pupil measures. We first …

Masked Face Analysis Via Multi-Task Deep Learning, Vatsa S. Patel, Zhongliang Nie, Trung-Nghia Le, Tam Van Nguyen

Computer Science Faculty Publications

Face recognition with wearable items has been a challenging task in computer vision and involves the problem of identifying humans wearing a face mask. Masked face analysis via multi-task learning could effectively improve performance in many fields of face analysis. In this paper, we propose a unified framework for predicting the age, gender, and emotions of people wearing face masks. We first construct FGNET-MASK, a masked face dataset for the problem. Then, we propose a multi-task deep learning model to tackle the problem. In particular, the multi-task deep learning model takes the data as inputs and shares their weight to …

Verification Of Piecewise Deep Neural Networks: A Star Set Approach With Zonotope Pre-Filter, Hoang-Dung Tran, Neelanjana Pal, Diego Manzanas Lopez, Patrick Musau, Xiaodong Yang, Luan Viet Nguyen, Weiming Xiang, Stanley Bak, Taylor T. Johnson

Computer Science Faculty Publications

Verification has emerged as a means to provide formal guarantees on learning-based systems incorporating neural network before using them in safety-critical applications. This paper proposes a new verification approach for deep neural networks (DNNs) with piecewise linear activation functions using reachability analysis. The core of our approach is a collection of reachability algorithms using star sets (or shortly, stars), an effective symbolic representation of high-dimensional polytopes. The star-based reachability algorithms compute the output reachable sets of a network with a given input set before using them for verification. For a neural network with piecewise linear activation functions, our approach can …

Olympic Games Event Recognition Via Transfer Learning With Photobombing Guided Data Augmentation, Yousef I. Mohamad, Samah S. Baraheem, Tam Van Nguyen

Computer Science Faculty Publications

Automatic event recognition in sports photos is both an interesting and valuable research topic in the field of computer vision and deep learning. With the rapid increase and the explosive spread of data, which is being captured momentarily, the need for fast and precise access to the right information has become a challenging task with considerable importance for multiple practical applications, i.e., sports image and video search, sport data analysis, healthcare monitoring applications, monitoring and surveillance systems for indoor and outdoor activities, and video captioning. In this paper, we evaluate different deep learning models in recognizing and interpreting the sport …

R2u3d: Recurrent Residual 3d U-Net For Lung Segmentation, Dhaval D. Kadia, Md Zahangir Alom, Ranga Burada, Tam Nguyen, Vijayan K. Asari

Computer Science Faculty Publications

3D Lung segmentation is essential since it processes the volumetric information of the lungs, removes the unnecessary areas of the scan, and segments the actual area of the lungs in a 3D volume. Recently, the deep learning model, such as U-Net outperforms other network architectures for biomedical image segmentation. In this paper, we propose a novel model, namely, Recurrent Residual 3D U-Net (R(2)U3D), for the 3D lung segmentation task. In particular, the proposed model integrates 3D convolution into the Recurrent Residual Neural Network based on U-Net. It helps learn spatial dependencies in 3D and increases the propagation of 3D volumetric …

Divide And Slide: Layer-Wise Refinement For Output Range Analysis Of Deep Neural Networks, Chao Huang, Jiameng Fan, Xin Chen, Wenchao Li, Qi Zhu

Computer Science Faculty Publications

In this article, we present a layer-wise refinement method for neural network output range analysis. While approaches such as nonlinear programming (NLP) can directly model the high nonlinearity brought by neural networks in output range analysis, they are known to be difficult to solve in general. We propose to use a convex polygonal relaxation (overapproximation) of the activation functions to cope with the nonlinearity. This allows us to encode the relaxed problem into a mixed-integer linear program (MILP), and control the tightness of the relaxation by adjusting the number of segments in the polygon. Starting with a segment number of …

Repurposing Visual Input Modalities For Blind Users: A Case Study Of Word Processors, Hae-Na Lee, Vikas Ashok, I.V. Ramakrishnan

Computer Science Faculty Publications

Visual 'point-and-click' interaction artifacts such as mouse and touchpad are tangible input modalities, which are essential for sighted users to conveniently interact with computer applications. In contrast, blind users are unable to leverage these visual input modalities and are thus limited while interacting with computers using a sequentially narrating screen-reader assistive technology that is coupled to keyboards. As a consequence, blind users generally require significantly more time and effort to do even simple application tasks (e.g., applying a style to text in a word processor) using only keyboard, compared to their sighted peers who can effortlessly accomplish the same tasks …

Rotate-And-Press: A Non-Visual Alternative To Point-And-Click, Hae-Na Lee, Vikas Ashok, I. V. Ramakrishnan

Computer Science Faculty Publications

Most computer applications manifest visually rich and dense graphical user interfaces (GUIs) that are primarily tailored for an easy-and-efficient sighted interaction using a combination of two default input modalities, namely the keyboard and the mouse/touchpad. However, blind screen-reader users predominantly rely only on keyboard, and therefore struggle to interact with these applications, since it is both arduous and tedious to perform the visual 'point-and-click' tasks such as accessing the various application commands/features using just keyboard shortcuts supported by screen readers.

In this paper, we investigate the suitability of a 'rotate-and-press' input modality as an effective non-visual substitute for the visual …

Nnv: The Neural Network Verification Tool For Deep Neural Networks And Learning-Enabled Cyber-Physical Systems, Hoang-Dung Tran, Xiaodong Yang, Diego Manzanas Lopez, Patrick Musau, Luan Viet Nguyen, Weiming Xiang, Stanley Bak, Taylor T. Johnson

Computer Science Faculty Publications

This paper presents the Neural Network Verification (NNV) software tool, a set-based verification framework for deep neural networks (DNNs) and learning-enabled cyber-physical systems (CPS). The crux of NNV is a collection of reachability algorithms that make use of a variety of set representations, such as polyhedra, star sets, zonotopes, and abstract-domain representations. NNV supports both exact (sound and complete) and over-approximate (sound) reachability algorithms for verifying safety and robustness properties of feed-forward neural networks (FFNNs) with various activation functions. For learning-enabled CPS, such as closed-loop control systems incorporating neural networks, NNV provides exact and over-approximate reachability analysis schemes for linear …

A Saliency-Driven Video Magnifier For People With Low Vision, Ali Selman Aydin, Shirin Feiz, Iv Ramakrishnan, Vikas Ashok

Computer Science Faculty Publications

Consuming video content poses significant challenges for many screen magnifier users, which is the “go to” assistive technology for people with low vision. While screen magnifier software could be used to achieve a zoom factor that would make the content of the video visible to low-vision users, it is oftentimes a major challenge for these users to navigate through videos. Towards making videos more accessible for low-vision users, we have developed the SViM video magnifier system [6]. Specifically, SViM consists of three different magnifier interfaces with easy-to-use means of interactions. All three interfaces are driven by visual saliency as a …

Towards Making Videos Accessible For Low Vision Screen Magnifier Users, Ali Selman Aydin, Shirin Feiz, Vikas Ashok, Iv Ramakrishnan

Computer Science Faculty Publications

People with low vision who use screen magnifiers to interact with computing devices find it very challenging to interact with dynamically changing digital content such as videos, since they do not have the luxury of time to manually move, i.e., pan the magnifier lens to different regions of interest (ROIs) or zoom into these ROIs before the content changes across frames.

In this paper, we present SViM, a first of its kind screen-magnifier interface for such users that leverages advances in computer vision, particularly video saliency models, to identify salient ROIs in videos. SViM's interface allows users to zoom in/out …

Sail: Saliency-Driven Injection Of Aria Landmarks, Ali Selman Aydin, Shirin Feiz, Vikas Ashok, Iv Ramakrishnan

Computer Science Faculty Publications

Navigating webpages with screen readers is a challenge even with recent improvements in screen reader technologies and the increased adoption of web standards for accessibility, namely ARIA. ARIA landmarks, an important aspect of ARIA, lets screen reader users access different sections of the webpage quickly, by enabling them to skip over blocks of irrelevant or redundant content. However, these landmarks are sporadically and inconsistently used by web developers, and in many cases, even absent in numerous web pages. Therefore, we propose SaIL, a scalable approach that automatically detects the important sections of a web page, and then injects ARIA landmarks …

Reachnn: Reachability Analysis Of Neural-Network Controlled Systems, Chao Huang, Jiameng Fan, Wenchao Li, Xin Chen, Qi Zhu

Computer Science Faculty Publications

Applying neural networks as controllers in dynamical systems has shown great promises. However, it is critical yet challenging to verify the safety of such control systems with neural-network controllers in the loop. Previous methods for verifying neural network controlled systems are limited to a few specific activation functions. In this work, we propose a new reachability analysis approach based on Bernstein polynomials that can verify neural-network controlled systems with a more general form of activation functions, i.e., as long as they ensure that the neural networks are Lipschitz continuous. Specifically, we consider abstracting feedforward neural networks with Bernstein polynomials for …

An Introduction To Declarative Programming In Clips And Prolog, Jack L. Watkin, Adam C. Volk, Saverio Perugini

Computer Science Faculty Publications

We provide a brief introduction to CLIPS—a declarative/logic programming language for implementing expert systems—and PROLOG—a declarative/logic programming language based on first-order, predicate calculus. Unlike imperative languages in which the programmer specifies how to compute a solution to a problem, in a declarative language, the programmer specifies what they what to find, and the system uses a search strategy built into the language. We also briefly discuss applications of CLIPS and PROLOG.

Impact Of Http Cookie Violations In Web Archives, Sawood Alam, Michele C. Weigle, Michael L. Nelson

Computer Science Faculty Publications

Certain HTTP Cookies on certain sites can be a source of content bias in archival crawls. Accommodating Cookies at crawl time, but not utilizing them at replay time may cause cookie violations, resulting in defaced composite mementos that never existed on the live web. To address these issues, we propose that crawlers store Cookies with short expiration time and archival replay systems account for values in the Vary header along with URIs.

Developing A Contemporary And Innovative Operating Systems Course, Saverio Perugini, David J. Wright

Computer Science Faculty Publications

This birds-of-a-feather provides a discussion forum to foster innovation in teaching operating systems (os) at the undergraduate level. This birds-of-a-feather seeks to generate discussion and ideas around pedagogy for os and, in particular, how we might develop a contemporary and innovative model, in both content and delivery, for an os course—that plays a central role in a cs curriculum—and addresses significant issues of misalignment between existing os courses and employee professional skills and knowledge requirements. We would like to exchange ideas regarding a re-conceptualized course model of os curriculum and related pedagogy, especially in the areas of mobile OSs and …

An Interactive, Graphical Simulator For Teaching Operating Systems, Joshua W. Buck, Saverio Perugini

Computer Science Faculty Publications

We demonstrate a graphical simulation tool for visually and interactively exploring the processing of a variety of events handled by an operating system when running a program. Our graphical simulator is available for use on the web by both instructors and students for purposes of pedagogy. Instructors can use it for live demonstrations of course concepts in class, while students can use it outside of class to explore the concepts. The graphical simulation tool is implemented using the React library for the fancy ui elements of the Node.js framework and is available as a web application at https://cpudemo.azurewebsites.net. The goals …

A New Way To Detect Cyberattacks Extracting Changes In Register Values From Radio-Frequency Side Channels, Ronald A. Riley, James T. Graham, Ryan M. Fuller, Rusty O. Baldwin, Ashwin Fisher

Computer Science Faculty Publications

The Internet of Things (IoT) and the Internet of Everything (IoE) have driven processors into nearly every powered de- vice, from thermostats to refrigerators to light bulbs. From a security perspective, the IoT and IoE create a new layer of sig- nals and systems that can provide insight into the internal opera- tions of a device via analog side channels. Our research focuses on leveraging these analog side channels in IoT/IoE processors to detect intrusions. Our goal is to defend against cyberattacks that insert malware into IoT devices by detecting deviations in the code running on their processors from known …

Predicting Public Opinion On Drug Legalization: Social Media Analysis And Consumption Trends, Farahnaz Golrooy Motlagh, Saeedeh Shekarpour, Amit Sheth, Krishnaprasad Thirunarayan, Michael L. Raymer

Computer Science Faculty Publications

In this paper, we focus on the collection and analysis of relevant Twitter data on a state-by-state basis for (i) measuring public opinion on marijuana legalization by mining sentiment in Twitter data and (ii) determining the usage trends for six distinct types of marijuana. We overcome the challenges posed by the informal and ungrammatical nature of tweets to analyze a corpus of 306,835 relevant tweets collected over the four-month period, preceding the November 2015 Ohio Marijuana Legalization ballot and the four months after the election for all states in the US. Our analysis revealed two key insights: (i) the people …

Reachability Analysis For Neural Feedback Systems Using Regressive Polynomial Rule Inference, Souradeep Dutta, Xin Chen, Sriram Sankaranarayanan

Computer Science Faculty Publications

We present an approach to construct reachable set overapproxi- mations for continuous-time dynamical systems controlled using neural network feedback systems. Feedforward deep neural net- works are now widely used as a means for learning control laws through techniques such as reinforcement learning and data-driven predictive control. However, the learning algorithms for these net- works do not guarantee correctness properties on the resulting closed-loop systems. Our approach seeks to construct overapproxi- mate reachable sets by integrating a Taylor model-based flowpipe construction scheme for continuous differential equations with an approach that replaces the neural network feedback law for a small subset of …

The Design Of An Emerging/Multi-Paradigm Programming Languages Course, Saverio Perugini

Computer Science Faculty Publications

We present the design of a new special topics course, Emerging/Multi-paradigm Languages, on the recent trend toward more dynamic, multi-paradigm languages. To foster course adoption, we discuss the design of the course, which includes language presentations/papers and culminating, ��inal projects/papers. The goal of this article is to inspire and facilitate course adoption.