Graphics and Human Computer Interfaces Commons^™

Developing A Contemporary Operating Systems Course, Saverio Perugini, David J. Wright

Computer Science Faculty Publications

The objective of this tutorial presentation is to foster innovation in the teaching of operating systems (os) at the undergraduate level as part of a three-year NSF-funded IUSE (Improving Undergraduate STEM Education) project titled “Engaged Student Learning: Reconceptualizing and Evaluating a Core Computer Science Course for Active Learning and STEM Student Success” (2017–2020).

An Application Of The Actor Model Of Concurrency In Python: A Euclidean Rhythm Music Sequencer, Daniel P. Prince, Saverio Perugini

Computer Science Faculty Publications

We present a real-time sequencer, implementing the Euclidean rhythm algorithm, for creative generation of drum sequences by musicians or producers. We use the Actor model of concurrency to simplify the communication required for interactivity and musical timing, and generator comprehensions and higher-order functions to simplify the implementation of the Euclidean rhythm algorithm. The resulting application sends Musical Instrument Digital Interface (MIDI) data interactively to another application for sound generation.

Chameleon: A Customizable Language For Teaching Programming Languages, Saverio Perugini, Jack L. Watkin

Computer Science Faculty Publications

ChAmElEoN is a programming language for teaching students the concepts and implementation of computer languages. We describe its syntax and semantics, the educational aspects involved in the implementation of a variety of interpreters for it, its malleability, and student feedback to inspire its use for teaching languages.

Natural Language, Mixed-Initiative Personal Assistant Agents, Joshua W. Buck, Saverio Perugini, Tam W. Nguyen

Computer Science Faculty Publications

The increasing popularity and use of personal voice assistant technologies, such as Siri and Google Now, is driving and expanding progress toward the long-term and lofty goal of using artificial intelligence to build human-computer dialog systems capable of understanding natural language. While dialog-based systems such as Siri support utterances communicated through natural language, they are limited in the flexibility they afford to the user in interacting with the system and, thus, support primarily action-requesting and information-seeking tasks. Mixed-initiative interaction, on the other hand, is a flexible interaction technique where the user and the system act as equal participants in an …

Unobtrusive And Extensible Archival Replay Banners Using Custom Elements, Sawood Alam, Mat Kelly, Michele C. Weigle, Michael L. Nelson

Computer Science Faculty Publications

We compare and contrast three different ways to implement an archival replay banner. We propose an implementation that utilizes Custom Elements and adds some unique behaviors, not common in existing archival replay systems, to enhance the user experience. Our approach has a minimal user interface footprint and resource overhead while still providing rich interactivity and extended on-demand provenance information about the archived resources.

A Reliable And Efficient Wireless Sensor Network System For Water Quality Monitoring, Dung Nguyen, Phu Huu Phung

Computer Science Faculty Publications

Wireless sensor networks (WSNs) are strongly useful to monitor physical and environmental conditions to provide realtime information for improving environment quality. However, deploying a WSN in a physical environment faces several critical challenges such as high energy consumption, and data loss.In this work, we have proposed a reliable and efficient environmental monitoring system in ponds using wireless sensor network and cellular communication technologies. We have designed a hardware and software ecosystem that can limit the data loss yet save the energy consumption of nodes. A lightweight protocol acknowledges data transmission among the nodes. Data are transmitted to the cloud using …

P4sinc – An Execution Policy Framework For Iot Services In The Edge, Phu Huu Phung, Hong-Linh Truong, Divya Teja Yasoju

Computer Science Faculty Publications

Internet of Things (IoT) services are increasingly deployed at the edge to access and control Things. The execution of such services needs to be monitored to provide information for security, service contract, and system operation management. Although different techniques have been proposed for deploying and executing IoT services in IoT gateways and edge servers, there is a lack of generic policy frameworks for instrumentation and assurance of various types of execution policies for IoT services. In this paper, we present P4SINC as an execution policy framework that covers various functionalities for IoT services deployed in software-defined machines in IoT infrastructures. …

Hybridguard: A Principal-Based Permission And Fine-Grained Policy Enforcement Framework For Web-Based Mobile Applications, Phu Huu Phung, Abhinav Mohanty, Rahul Rachapalli, Meera Sridhar

Computer Science Faculty Publications

Web-based or hybrid mobile applications (apps) are widely used and supported by various modern hybrid app development frameworks. In this architecture, any JavaScript code, local or remote, can access available APIs, including JavaScript bridges provided by the hybrid framework, to access device resources. This JavaScript inclusion capability is dangerous, since there is no mechanism to determine the origin of the code to control access, and any JavaScript code running in the mobile app can access the device resources through the exposed APIs. Previous solutions are either limited to a particular platform (e.g., Android) or a specific hybrid framework (e.g., Cordova) …

Toward An Iot-Based Expert System For Heart Disease Diagnosis, Do Thanh Thai, Quang Tran Minh, Phu Huu Phung

Computer Science Faculty Publications

IoT technology has been recently adopted in the healthcare system to collect Electrocardiogram (ECG) signals for heart disease diagnosis and prediction. However, noises in collected ECG signals make the diagnosis and prediction system unreliable and imprecise. In this work, we have proposed a new lightweight approach to removing noises in collected ECG signals to perform precise diagnosis and prediction. First, we have used a revised Sequential Recursive (SR) algorithm to transform the signals into digital format. Then, the digital data is proceeded using a revised Discrete Wavelet Transform (DWT) algorithm to detect peaks in the data to remove noises. Finally, …

Static Human Detection And Scenario Recognition Via Wearable Thermal Sensing System, Qingquan Sun, Ju Shen, Haiyan Qiao, Xinlin Huang, Chen Chen, Fei Hu

Computer Science Faculty Publications

Conventional wearable sensors are mainly used to detect the physiological and activity information of individuals who wear them, but fail to perceive the information of the surrounding environment. This paper presents a wearable thermal sensing system to detect and perceive the information of surrounding human subjects. The proposed system is developed based on a pyroelectric infrared sensor. Such a sensor system aims to provide surrounding information to blind people and people with weak visual capability to help them adapt to the environment and avoid collision. In order to achieve this goal, a low-cost, low-data-throughput binary sampling and analyzing scheme is …

Cse: U: Mixed-Initiative Personal Assistant Agents, Joshua W. Buck, Saverio Perugini, Tam Nguyen

Computer Science Faculty Publications

Specification and implementation of flexible human-computer dialogs is challenging because of the complexity involved in rendering the dialog responsive to a vast number of varied paths through which users might desire to complete the dialog. To address this problem, we developed a toolkit for modeling and implementing task-based, mixed-initiative dialogs based on metaphors from lambda calculus. Our toolkit can automatically operationalize a dialog that involves multiple prompts and/or sub-dialogs, given a high-level dialog specification of it. The use of natural language with the resulting dialogs makes the flexibility in communicating user utterances commensurate with that in dialog completion paths—an aspect …

Homeomorphic Tetrahedral Tessellation For Biomedical Images, Jing Xu, Andrey N. Chernikov

Computer Science Faculty Publications

We present a novel algorithm for generating three-dimensional unstructured tetrahedral meshes for biomedical images. The method uses an octree as the background grid from which to build the final graded conforming meshes. The algorithm is fast and robust. It produces meshes with high quality since it provides dihedral angle lower bound for the output tetrahedra. Moreover, the mesh boundary is a geometrically and topologically accurate approximation of the object surface in the sense that it allows for guaranteed bounds on the two-sided Hausdorff distance and the homeomorphism between the boundaries of the mesh and the boundaries of the materials. The …

An Iterative Bézier Method For Fitting Beta-Sheet Component Of A Cryo-Em Density Map, Michael Poteat, Jing He

Computer Science Faculty Publications

Cryo-electron microscopy (Cryo-EM) is a powerful technique to produce 3-dimensional density maps for large molecular complexes. Although many atomic structures have been solved from cryo-EM density maps, it is challenging to derive atomic structures when the resolution of density maps is not sufficiently high. Geometrical shape representation of secondary structural components in a medium-resolution density map enhances modeling of atomic structures. We compare two methods in producing surface representation of the β-sheet component of a density map. Given a 3-dimensional volume of β-sheet that is segmented from a density map, the performance of a polynomial fitting was compared with that …

Marim: Mobile Augmented Reality For Interactive Manuals, Tam Nguyen, Dorothy Tan, Bilal Mirza, Jose Sepulveda

Computer Science Faculty Publications

In this work, we present a practical system which uses mobile devices for interactive manuals. In particular, there are two modes provided in the system, namely, expert/trainer and trainee modes. Given the expert/trainer editor, experts design the step-by-step interactive manuals. For each step, the experts capture the images by using phones/tablets and provide visual instructions such as interest regions, text, and action animations. In the trainee mode, the system utilizes the existing object detection and tracking algorithms to identify the step scene and retrieve the respective instruction to be displayed on the mobile device. The trainee then follows the displayed …

Leveraging Static Analysis Tools For Improving Usability Of Memory Error Sanitization Compilers, Rigel Gjomemo, Phu Huu Phung, Edmund Ballou, Kedar S. Namjoshi, V. N. Venkatakrishnan, Lenore Zuck

Computer Science Faculty Publications

Memory errors such as buffer overruns are notorious security vulnerabilities. There has been considerable interest in having a compiler to ensure the safety of compiled code either through static verification or through instrumented runtime checks. While certifying compilation has shown much promise, it has not been practical, leaving code instrumentation as the next best strategy for compilation. We term such compilers Memory Error Sanitization Compilers (MESCs). MESCs are available as part of GCC, LLVM and MSVC suites. Due to practical limitations, MESCs typically apply instrumentation indiscriminately to every memory access, and are consequently prohibitively expensive and practical to only small …

A Language-Based Model For Specifying And Staging Mixed-Initiative Dialogs, Saverio Perugini, Joshua W. Buck

Computer Science Faculty Publications

Specifying and implementing flexible human-computer dialogs, such as those used in kiosks, is complex because of the numerous and varied directions in which each user might steer a dialog. The objective of this research is to improve dialog specification and implementation. To do so we developed a model for specifying and staging mixed-initiative dialogs. The model involves a dialog authoring notation, based on concepts from programming languages, for specifying a variety of unsolicited reporting, mixed-initiative dialogs in a concise representation that serves as a design for dialog implementation. Guided by this foundation, we built a dialog staging engine which operationalizes …

Gecka3d: A 3d Game Engine For Commonsense Knowledge Acquisition, Erik Cambria, Tam Nguyen, Brian Cheng, Kenneth Kwok, Jose Sepulveda

Computer Science Faculty Publications

Commonsense knowledge representation and reasoning is key for tasks such as artificial intelligence and natural language understanding. Since commonsense consists of information that humans take for granted, gathering it is an extremely difficult task. In this paper, we introduce a novel 3D game engine for commonsense knowledge acquisition (GECKA3D) which aims to collect commonsense from game designers through the development of serious games. GECKA3D integrates the potential of serious games and games with a purpose. This provides a platform for the acquisition of reusable and multi-purpose knowledge and also enables the development of games that can provide entertainment value and …

Results And Challenges In Visualizing Analytic Provenance Of Text Analysis Tasks Using Interaction Logs, Rhema Linder, Alyssa M. Peña, Sampath Jayarathna, Eric D. Ragan

Computer Science Faculty Publications

After data analysis, recalling and communicating the steps and rationale followed during the analysis can be difficult. This paper explores the use of interaction logs to generate summaries of an analyst's interest based on interactions with specific data items in a text analysis scenario. Our approach uses data-interaction events as a proxy for user interest in and experience of information. Logging can produce verbose logs that detail all available readable content, so the discussed approach uses topic modeling (LDA) over different time segments to summarize the verbose information and generate visualizations of the history of user interest. Our preliminary results …

Revisiting The Futamura Projections: A Diagrammatic Approach, Brandon Williams, Saverio Perugini

Computer Science Faculty Publications

The advent of language implementation tools such as PyPy and Truffle/Graal have reinvigorated and broadened interest in topics related to automatic compiler generation and optimization. Given this broader interest, we revisit the Futamura Projections using a novel diagram scheme. Through these diagrams we emphasize the recurring patterns in the Futamura Projections while addressing their complexity and abstract nature. We anticipate that this approach will improve the accessibility of the Futamura Projections and help foster analysis of those new tools through the lens of partial evaluation.

Salad: A Multimodal Approach For Contextual Video Advertising, Chen Xiang, Tam Nguyen, Mohan Kankanhalli

Computer Science Faculty Publications

The explosive growth of multimedia data on Internet has created huge opportunities for online video advertising. In this paper, we propose a novel advertising technique called SalAd, which utilizes textual information, visual content and the webpage saliency, to automatically associate the most suitable companion ads with online videos. Unlike most existing approaches that only focus on selecting the most relevant ads, SalAd further considers the saliency of selected ads to reduce intentional ignorance. SalAd consists of three basic steps. Given an online video and a set of advertisements, we first roughly identify a set of relevant ads based on the …

An Immersive Telepresence System Using Rgb-D Sensors And Head-Mounted Display, Xinzhong Lu, Ju Shen, Saverio Perugini, Jianjun Yang

Computer Science Faculty Publications

We present a tele-immersive system that enables people to interact with each other in a virtual world using body gestures in addition to verbal communication. Beyond the obvious applications, including general online conversations and gaming, we hypothesize that our proposed system would be particularly beneficial to education by offering rich visual contents and interactivity. One distinct feature is the integration of egocentric pose recognition that allows participants to use their gestures to demonstrate and manipulate virtual objects simultaneously. This functionality enables the instructor to effectively and efficiently explain and illustrate complex concepts or sophisticated problems in an intuitive manner. The …

Visualizing Fantasy Fiction: Design Of A Class In Digital Scholarship And Visualization, Including Research, Organization And Digital Visualization, That Does Not Require Programming Or It Support, Charles W. Kann

Computer Science Faculty Publications

This paper outlines a course to integrate digital visualizations into undergraduate research. These visualizations will include mapping and timelines of events, and the ability to hyperlink the events, characters, and story lines in a fantasy fiction story such as Lord of the Rings or A Game of Thrones. The digital scholarship will involve the methodology for collecting, organizing, and representing the data for the visualizations.

The topic for the visualizations in this paper is fantasy fiction; however the methods to develop these visualizations will be applicable to many academic disciplines, including the humanities and social sciences.

The paper outlines …

Sense Beyond Expressions: Cuteness, Kang Wang, Tam Nguyen, Jiashi Feng, Jose Sepulveda

Computer Science Faculty Publications

With the development of Internet culture, cute has become a popular concept. Many people are curious about what factors making a person look cute. However, there is rare research to answer this interesting question. In this work, we construct a dataset of personal images with comprehensively annotated cuteness scores and facial attributes to investigate this high-level concept in depth. Based on this dataset, through an automatic attributes mining process, we find several critical attributes determining the cuteness of a person.

We also develop a novel Continuous Latent Support Vector Machine (C-LSVM) method to predict the cuteness score of one person …

Adaptive Nonparametric Image Parsing, Tam Nguyen, Canyi Lu, Jose Sepulveda, Shuicheng Yan

Computer Science Faculty Publications

In this paper, we present an adaptive nonparametric solution to the image parsing task, namely, annotating each image pixel with its corresponding category label. For a given test image, first, a locality-aware retrieval set is extracted from the training data based on superpixel matching similarities, which are augmented with feature extraction for better differentiation of local superpixels. Then, the category of each superpixel is initialized by the majority vote of the k -nearest-neighbor superpixels in the retrieval set. Instead of fixing k as in traditional nonparametric approaches, here, we propose a novel adaptive nonparametric approach that determines the sample-specific k …

Teaching Big Data By Three Levels Of Projects, Jianjun Yang, Ju Shen

Computer Science Faculty Publications

Big Data is a new topic and it is very hot nowadays. However, it is difficult to teach Big Data effectively by regular lecture. In this paper, we present a unique way to teach students Big Data by developing three levels of projects from easy to difficult. The three levels projects are initializing project, designing project, and comprehensive projects. They are developed to involve students in Big Data, train students' skills to analyze concrete problems of Big Data, and develop students' creative abilities and their abilities to solve real setting problems.

Automatic Video Self Modeling For Voice Disorder, Ju Shen, Changpeng Ti, Anusha Raghunathan, Sen-Ching S. Cheung, Rita Patel

Computer Science Faculty Publications

Video self modeling (VSM) is a behavioral intervention technique in which a learner models a target behavior by watching a video of him- or herself. In the field of speech language pathology, the approach of VSM has been successfully used for treatment of language in children with Autism and in individuals with fluency disorder of stuttering. Technical challenges remain in creating VSM contents that depict previously unseen behaviors. In this paper, we propose a novel system that synthesizes new video sequences for VSM treatment of patients with voice disorders. Starting with a video recording of a voice-disorder patient, the proposed …

Salient Object Detection Via Augmented Hypotheses, Tam Nguyen, Jose Sepulveda

Computer Science Faculty Publications

In this paper, we propose using augmented hypotheses which consider objectness, foreground, and compactness for salient object detection. Our algorithm consists of four basic steps. First, our method generates the objectness map via objectness hypotheses. Based on the objectness map, we estimate the foreground margin and compute the corresponding foreground map which prefers the foreground objects. From the objectness map and the foreground map, the compactness map is formed to favor the compact objects. We then derive a saliency measure that produces a pixel-accurate saliency map which uniformly covers the objects of interest and consistently separates foreground and background.

We …

Between Worlds: Securing Mixed Javascript/Actionscript Multi-Party Web Content, Phu Huu Phung, Maliheh Monshizadeh, Meera Sridhar, Kevin W. Hamlen, V. N. Venkatakrishnan

Computer Science Faculty Publications

Mixed Flash and JavaScript content has become increasingly prevalent; its purveyance of dynamic features unique to each platform has popularized it for myriad Web development projects. Although Flash and JavaScript security has been examined extensively, the security of untrusted content that combines both has received considerably less attention. This article considers this fusion in detail, outlining several practical scenarios that threaten the security of Web applications. The severity of these attacks warrants the development of new techniques that address the security of Flash-JavaScript content considered as a whole, in contrast to prior solutions that have examined Flash or JavaScript security …

Compression Of Video Tracking And Bandwidth Balancing Routing In Wireless Multimedia Sensor Networks, Yin Wang, Jianjun Yang, Ju Shen, Bryson Payne, Juan Guo, Kun Hua

Computer Science Faculty Publications

There has been a tremendous growth in multimedia applications over wireless networks. Wireless Multimedia Sensor Networks(WMSNs) have become the premier choice in many research communities and industry. Many state-of-art applications, such as surveillance, traffic monitoring, and remote heath care are essentially video tracking and transmission in WMSNs. The transmission speed is constrained by the big file size of video data and fixed bandwidth allocation in constant routing paths. In this paper, we present a CamShift based algorithm to compress the tracking of videos. Then we propose a bandwidth balancing strategy in which each sensor node is able to dynamically select …

Leading Undergraduate Students To Big Data Generation, Jianjun Yang, Ju Shen

Computer Science Faculty Publications

People are facing a flood of data today. Data are being collected at unprecedented scale in many areas, such as networking, image processing, virtualization, scientific computation, and algorithms. The huge data nowadays are called Big Data. Big data is an all encompassing term for any collection of data sets so large and complex that it becomes difficult to process them using traditional data processing applications. In this article, the authors present a unique way which uses network simulator and tools of image processing to train students abilities to learn, analyze, manipulate, and apply Big Data. Thus they develop students hands-on …