Graphics and Human Computer Interfaces Commons^™

A Language-Based Model For Specifying And Staging Mixed-Initiative Dialogs, Saverio Perugini, Joshua W. Buck

Computer Science Faculty Publications

Specifying and implementing flexible human-computer dialogs, such as those used in kiosks, is complex because of the numerous and varied directions in which each user might steer a dialog. The objective of this research is to improve dialog specification and implementation. To do so we developed a model for specifying and staging mixed-initiative dialogs. The model involves a dialog authoring notation, based on concepts from programming languages, for specifying a variety of unsolicited reporting, mixed-initiative dialogs in a concise representation that serves as a design for dialog implementation. Guided by this foundation, we built a dialog staging engine which operationalizes …

Event Detection With Zero Example: Select The Right And Suppress The Wrong Concepts, Yi-Jie Lu, Hao Zhang, Maaike De Boer, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Complex video event detection without visual examples is a very challenging issue in multimedia retrieval. We present a state-of-the-art framework for event search without any need of exemplar videos and textual metadata in search corpus. To perform event search given only query words, the core of our framework is a large, pre-built bank of concept detectors which can understand the content of a video in the perspective of object, scene, action and activity concepts. Leveraging such knowledge can effectively narrow the semantic gap between textual query and the visual content of videos. Besides the large concept bank, this paper focuses …

Serendipity-Driven Celebrity Video Hyperlinking, Shujun Yang, Lei Pang, Chong-Wah Ngo, Benoit Huet

Research Collection School Of Computing and Information Systems

This demo showcases the utility of video hyperlinks with celebrities as the link anchors and their social circles as targets, aiming to help users quickly explore the aboutness of a celebrity by link traversal. Through content analysis, our system embeds hyperlinks into videos such that users can click-and-jump between celebrity faces in different videos to get-to-know their social circles. One peculiar feature is the ability of the system in providing links that maximize users' chance encounter, or serendipitous experience, beyond information need. Our system is enabled by two key components, name-face association and diversity-based ranking, for the aboutness and serendipity …

Video Modeling And Learning On Riemannian Manifold For Emotion Recognition In The Wild, Mengyi Liu, Ruiping Wang, Shaoxin Li, Zhiwu Huang, Shiguang Shan, Xilin Chen

Research Collection School Of Computing and Information Systems

In this paper, we present the method for our submission to the emotion recognition in the wild challenge (EmotiW). The challenge is to automatically classify the emotions acted by human subjects in video clips under real-world environment. In our method, each video clip can be represented by three types of image set models (i.e. linear subspace, covariance matrix, and Gaussian distribution) respectively, which can all be viewed as points residing on some Riemannian manifolds. Then different Riemannian kernels are employed on these set models correspondingly for similarity/ distance measurement. For classification, three types of classifiers, i.e. kernel SVM, logistic regression, …

Exemplar-Driven Top-Down Saliency Detection Via Deep Association, Shengfeng He, Rynson W. H. Lau, Qingxiong Yang

Research Collection School Of Computing and Information Systems

Top-down saliency detection is a knowledge-driven search task. While some previous methods aim to learn this "knowledge" from category-specific data, others transfer existing annotations in a large dataset through appearance matching. In contrast, we propose in this paper a locateby-exemplar strategy. This approach is challenging, as we only use a few exemplars (up to 4) and the appearances among the query object and the exemplars can be very different. To address it, we design a two-stage deep model to learn the intra-class association between the exemplars and query objects. The first stage is for learning object-to-object association, and the second …

Visual Code: Breaking The Binary, Jacob Johannesen, Andrew Adriance

Liberal Arts and Engineering Studies

This project seeks to create an accessible programming language that is more visually based. Although some solutions exist, namely MIT’s Scratch, nothing has caught up to the mobile age. This proj- ect aims to reframe creating a game or app into the context of tell- ing a story, putting character creation first. By researching sto- ry-telling and how people learn, and by applying technical and user interface design knowledge, this project intends to deliver a soft- ware solution that opens introductory coding education to more people.

Hci Testing In Laboratory Or Field Settings, Chuan-Hoo Tan, Austin Silva, Rich Lee, Kanliang Wang, Fiona Fui-Hoon Nah

Research Collection School Of Computing and Information Systems

This paper presents perspectives from both academia and practice on how an HCI testing is to be conducted and the deliberations that go into the testing. HCI testing can be conducted in closed-door laboratory or in a field setting. While there is an increased interest in field testing of an HCI artifact, there is always an enduring concern over how to administer a field testing given that the testers will have less control over the course of testing. In this paper, we cover HCI testing deliberation as well as the operational issues of field testing, and conclude the paper with …

Patient Engagement In The Medical Facility Waiting Room Using Gamified Healthcare Information Delivery, Raheel Hassan, Nathan W. Twyman, Fiona Fui-Hoon Nah, Keng Siau

Research Collection School Of Computing and Information Systems

This study explores the proposition that medical facility waiting rooms are an opportune setting to engage with and educate patients while they are waiting for care. In collaboration with emergency department (ED) personnel, we developed ER Hero, a tablet-based application for waiting rooms that introduces patients to ED professionals and operations through mini-games and story-like interaction. We evaluated this prototype with human participants to determine how well it performed when compared to paper-based information disclosure presenting the same information. Participants using the application exhibited increased ED knowledge, decreased nervousness, and increased interest. The gamified application outperformed a paper-based approach on …

Smart Living For Elderly: Design And Human-Computer Interaction Considerations, Ranjana Sharma, Fiona Fui-Hoon Nah, Kavya Sharma, Teja S. Katta, Natalie Pang, Alvin Yong

Research Collection School Of Computing and Information Systems

To address aging challenges, we examine the concept of smart living and its applications for the elderly. Smart living refers to improving quality of life by transforming environments to become more intelligent and adaptable to users. In this paper, we discuss how smart living applications can help to address the needs of the elderly, as well as the design and human-computer interaction considerations for such applications.

Texture Modelling Using Convolutional Neural Networks, Leon A. Gatys, Alexander S. Ecker, Matthias Bethge

MODVIS Workshop

We introduce a new model of natural textures based on the feature spaces of convolutional neural networks optimised for object recognition. Samples from the model are of high perceptual quality demonstrating the generative power of neural networks trained in a purely discriminative fashion. Within the model, textures are represented by the correlations between feature maps in several layers of the network. We show that across layers the texture representations increasingly capture the statistical properties of natural images while making object information more and more explicit. Extending this framework to texture transfer, we introduce A Neural Algorithm of Artistic Style that …

2016-01-A3dsrinp-Csc-Sta-Cmb-522-Bps-542, Raymond Pulver, Neal Buxton, Xiaodong Wang, John Lucci, Jean Yves Hervé, Lenore Martin

Bioinformatics Software Design Projects

Cholesterol is carried and transported through bloodstream by lipoproteins. There are two types of lipoproteins: low density lipoprotein, or LDL, and high density lipoprotein, or HDL. LDL cholesterol is considered “bad” cholesterol because it can form plaque and hard deposit leading to arteries clog and make them less flexible. Heart attack or stroke will happen if the hard deposit blocks a narrowed artery. HDL cholesterol helps to remove LDL from the artery back to the liver.

Traditionally, particle counts of LDL and HDL plays an important role to understanding and prediction of heart disease risk. But recently research suggested that …

Augmented Reality In The Classroom, Patrick Jb Foster, Sean Cunniff

Honors Thesis

Low vision can have an exceptionally negative impact on a student’s ability to learn, especially when subjected to the conventional education system. In this environment, students are expected to adhere to a lecture that delivers most information visually via a whiteboard or a projector screen. The goal of this project is to create a customizable application for a smartphone that implements selective processing in order to make it easier for visually impaired students to engage with and learn from lectures.

Specifically, this application is written in the Java language for the Android platform. The application uses OpenGL ES, a C-like …

Improving Electroencephalography-Based Imagined Speech Recognition With A Simultaneous Video Data Stream, Sarah J. Stolze

Computer Science and Computer Engineering Undergraduate Honors Theses

Electroencephalography (EEG) devices offer a non-invasive mechanism for implementing imagined speech recognition, the process of estimating words or commands that a person expresses only in thought. However, existing methods can only achieve limited predictive accuracy with very small vocabularies; and therefore are not yet sufficient to enable fluid communication between humans and machines. This project proposes a new method for improving the ability of a classifying algorithm to recognize imagined speech recognition, by collecting and analyzing a large dataset of simultaneous EEG and video data streams. The results from this project suggest confirmation that complementing high-dimensional EEG data with similarly …

Gecka3d: A 3d Game Engine For Commonsense Knowledge Acquisition, Erik Cambria, Tam Nguyen, Brian Cheng, Kenneth Kwok, Jose Sepulveda

Computer Science Faculty Publications

Commonsense knowledge representation and reasoning is key for tasks such as artificial intelligence and natural language understanding. Since commonsense consists of information that humans take for granted, gathering it is an extremely difficult task. In this paper, we introduce a novel 3D game engine for commonsense knowledge acquisition (GECKA3D) which aims to collect commonsense from game designers through the development of serious games. GECKA3D integrates the potential of serious games and games with a purpose. This provides a platform for the acquisition of reusable and multi-purpose knowledge and also enables the development of games that can provide entertainment value and …

Enabling Usage Pattern-Based Logical Status Inference For Mobile Phones, Jon C. Hammer

Graduate Theses and Dissertations

Logical statuses of mobile users, such as isBusy and isAlone, are the key enabler for a plethora of context-aware mobile applications. While on-board hardware sensors (such as motion, proximity, and location sensors) have been extensively studied for logical status inference, continuous usage typically requires formidable energy consumption, which degrades the user experience. In this thesis, we argue that smartphone usage statistics can be used for logical status inference with negligible energy cost. To validate this argument, we present a continuous inference engine that (1) intercepts multiple operating system events, in particular foreground app, notifications, screen states, and connected networks; (2) …

Modeling Human-Like Non-Rationality For Social Agents, Jaroslaw Kochanowicz, Ah-Hwee Tan, Daniel Thalmann

Research Collection School Of Computing and Information Systems

Humans are not rational beings. Deviations from rationality in human thinking are currently well documented [25] as non-reducible to rational pursuit of egoistic benefit or its occasional distortion with temporary emotional excitation, as it is often assumed. This occurs not only outside conceptual reasoning or rational goal realization but also subconsciously and often in certainty that they did not and could not take place ‘in my case’. Non-rationality can no longer be perceived as a rare affective abnormality in otherwise rational thinking, but as a systemic, permanent quality, ’a design feature’ of human cognition. While social psychology has systematically addressed …

An Autonomous Agent For Learning Spatiotemporal Models Of Human Daily Activities, Shan Gao, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

Activities of Daily Living (ADLs) refer to activities performed by individuals on a daily basis. As ADLs are indicatives of a person’s habits, lifestyle, and well being, learning the knowledge of people’s ADL routine has great values in the healthcare and consumer domains. In this paper, we propose an autonomous agent, named Agent for Spatia-Temporal Activity Pattern Modeling (ASTAPM), being able to learn spatial and temporal patterns of human ADLs. ASTAPM utilises a self-organizing neural network model named Spatiotemporal - Adaptive Resonance Theory (ST-ART). ST-ART is capable of integrating multimodal contextual information, involving the time and space, wherein the ADL …

You Are Being Watched: Bystanders' Perspective On The Use Of Camera Devices In Public Spaces, Samarth Singhal, Carman Neustaedter, Thecla Schiphorst, Anthony Tang, Abhisekh Patra, Rui Pan

Research Collection School Of Computing and Information Systems

We are observing an increase in the use of smartphones and wearable devices in public places for streaming and recording video. Yet the use of cameras in these devices can infringe upon the privacy of the people in the surrounding environment by inadvertently capturing them. This paper presents findings from an in-situ exploratory study that investigates bystanders' reactions and feelings towards streaming and recording videos with smartphones and wearable glasses in public spaces. We use the interview results to guide an exploration of design directions for mobile video.

Efficient 3d Dental Identification Via Signed Feature Histogram And Learning Keypoint Detection, Zhiyuan Zhang, Sim Heng Ong, Xin Zhong, Kelvin W. C. Foong

Research Collection School Of Computing and Information Systems

Current methods of dental identification are mainly based on 2D dental radiographs which suffer from speed and accuracy limitations. In this paper, we present an efficient dental identification approach based on 3D dental models. We propose a novel shape descriptor, the Signed Feature Histogram (SFH), which is highly discriminative and can be easily computed to describe the local surface. Based on the SFH, a learning keypoint detection method is adopted to accurately detect the desired keypoints on both antemortem (AM) and postmortem (PM) models. For a given PM model, the optimal initial alignment to the AM model to be matched …

An Exploration Of The Effectiveness Of The Use Of Communication Apps Through Mobile Devices On Children With Autism Spectrum Disorders (Asd), Miriam O Sullivan

Theses

Autism Spectrum Disorder (ASD) is a complex neurological disorder which impacts on people in three primary areas: communication skills, social skills and behaviour skills. The diagnosis of ASD continues to rise with an estimated 1 in 100 receiving diagnoses of ASD in Ireland. There are many interventions promoted on a regular basis that claim positive effects on the basis of empirical research. However, the claims made by such studies are sometimes a little at odds with the level of methodological rigour and/or sample size employed. The research available is based on small scale international studies (America, Canada and Australia); however, …