Graphics and Human Computer Interfaces Commons^™

Nextgen Flight Deck Data Comm: Auxiliary Synthetic Speech Phase I, Tracy Lennertz, Judith Bürki-Cohen, Andrea L. Sparko, Nickolas D. Macchiarella, Jason Kring, Mike Coman, Tom Haritos, Jeffry Alvarado

Publications

Data Comm—a text-based controller-pilot communication system—is critical to many NextGen improvements. With Data Comm, communication becomes a visual task. Interacting with a visual Data Comm display may yield an unsafe increase in head-down time, particularly for single-pilot operations. This study examined the feasibility of supplementing Data Comm with synthetic speech. To this end, thirty-two pilots flew two experimental scenarios in a Cessna 172 Flight Training Device. In one scenario, ATC communication was with a text-only Data Comm display, in the other, communication was with a text Data Comm display with synthetic speech that read aloud each message (i.e., text+speech). Pilots …

Electronic Device, Display And Touch-Sensitive User Interface (Dec 2012), Michael E. Miller, Jerald J. Muszak, Michael J. Telek

AFIT Patents

Display devices and methods for operating the same are provided. In one embodiment, the display device has an electronic display having an active area for presenting visual content; a housing holding the electronic display and having an opening allowing a person to view a first portion of the active area; and a bezel about the opening, the bezel covering a second portion of the active area and providing a window through which at least a part of the second portion can be viewed. A sensor system senses when a person is close to touching the bezel or when a person …

The Design And Implementation Of A Mobile Game Engine For The Android Platform, Jon Hammer

Computer Science and Computer Engineering Undergraduate Honors Theses

In this thesis, a two-dimensional game engine is proposed for the Android mobile platform that facilitates rapid development of those games by individual developers or hobbyists. The essential elements of game design are presented so as to introduce the reader to the concepts that are crucial for comprehension of the paper. A brief overview of the Android Operating System is also included for those unfamiliar with it. Three primary design goals are identified, and a prototype solution is described in detail. The prototype is then evaluated against those design goals to see how well it accomplishes each task. The results …

Investigating Intelligent Agents In A 3d Virtual World, Yilin Kang, Fiona Fui-Hoon Nah, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

Web 3.0 involves " intelligent " web applications that utilize natural language processing, machine-based learning and reasoning, and intelligent techniques to analyze and understand user behavior. In this research, we empirically assess a specific form of Web 3.0 application in the form of intelligent agents that offer assistance to users in the virtual world. Using media naturalness theory, we hypothesize that the use of intelligent agents in the virtual world can enhance user experience by offering a more natural way of communication and assistance to users. We are interested to test if media naturalness theory holds in the context of …

Agent-Based Virtual Humans In Co-Space: An Evaluative Study, Yilin Kang, Ah-Hwee Tan, Fiona Fui-Hoon Nah

Research Collection School Of Computing and Information Systems

Co-Space refers to interactive virtual environment modelled after the real world in terms of look-and-feel, functionalities and services. We have developed a 3D virtual world named Nanyang Technological University (NTU) CoSpace populated with virtual human characters. Three key requirements of realistic virtual humans in the virtual world have been identified, namely (1) autonomy: agents can function on their own; (2) interactivity: agents can interact naturally with players; and (3) personality: agents can exhibit human traits and characteristics. Working towards these challenges, we propose a brain-inspired agent architecture that integrates goal-directed autonomy, natural language interaction and human-like personality. We conducted an …

Eliciting Usable Gestures For Multi-Display Environments, Teddy Seyed, Chris Burns, Mario Costa Sousa, Frank Maurer, Anthony Tang

Research Collection School Of Computing and Information Systems

Multi-display environments (MDEs) have advanced rapidly in recent years, incorporating multi-touch tabletops, tablets, wall displays and even position tracking systems. Designers have proposed a variety of interesting gestures for use in an MDE, some of which involve a user moving their hands, arms, body or even a device itself. These gestures are often used as part of interactions to move data between the various components of an MDE, which is a longstanding research problem. But designers, not users, have created most of these gestures and concerns over implementation issues such as recognition may have influenced their design. We performed a …

Electronic Device, Display And Touch-Sensitive User Interface (Nov 2012), Michael E. Miller, Jerald J. Muszak, Michael J. Telek

AFIT Patents

Display devices and methods for operating the same are provided. In one embodiment, the display device has an electronic display having an active area for presenting visual content; a housing holding the electronic display and having an opening allowing a person to view a first portion of the active area; and a bezel about the opening, the bezel covering a second portion of the active area and providing a window through which at least a part of the second portion can be viewed. A sensor system senses when a person is close to touching the bezel or when a person …

Cross-View Graph Embedding, Zhiwu Huang, S. Shan, H. Zhang, S. Lao, X. Chen

Research Collection School Of Computing and Information Systems

Recently, more and more approaches are emerging to solve the cross-view matching problem where reference samples and query samples are from different views. In this paper, inspired by Graph Embedding, we propose a unified framework for these cross-view methods called Cross-view Graph Embedding. The proposed framework can not only reformulate most traditional cross-view methods (e.g., CCA, PLS and CDFE), but also extend the typical single-view algorithms (e.g., PCA, LDA and LPP) to cross-view editions. Furthermore, our general framework also facilitates the development of new cross-view methods. In this paper, we present a new algorithm named Cross-view Local Discriminant Analysis (CLODA) …

Snap-And-Ask: Answering Multimodal Question By Naming Visual Instance, Wei Zhang, Lei Pang, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

In real-life, it is easier to provide a visual cue when asking a question about a possibly unfamiliar topic, for example, asking the question, “Where was this crop circle found?”. Providing an image of the instance is far more convenient than texting a verbose description of the visual properties, especially when the name of the query instance is not known. Nevertheless, having to identify the visual instance before processing the question and eventually returning the answer makes multimodal question-answering technically challenging. This paper addresses the problem of visual-totext naming through the paradigm of answering-by-search in a two-stage computational framework, which …

Predicting Domain Adaptivity: Redo Or Recycle?, Ting Yao, Chong-Wah Ngo, Shiai Zhu

Research Collection School Of Computing and Information Systems

Over the years, the academic researchers have contributed various visual concept classifiers. Nevertheless, given a new dataset, most researchers still prefer to develop large number of classifiers from scratch despite expensive labeling efforts and limited computing resources. A valid question is why not multimedia community “embrace the green” and recycle off-the-shelf classifiers for new dataset. The difficulty originates from the domain gap that there are many different factors that govern the development of a classifier and eventually drive its performance to emphasize certain aspects of dataset. Reapplying a classifier to an unseen dataset may end up GIGO (garbage in, garbage …

Fashionask: Pushing Community Answers To Your Fingertips, Wei Zhang, Lei Pang, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

We demonstrate a multimedia-based question-answering system, named FashionAsk, by allowing users to ask questions referring to pictures snapped by mobile devices. Specifically, instead of asking verbose questions to depict visual instances, direct pictures are provided as part of questions. To answer these multi-modal questions, FashionAsk performs a large-scale instance search to infer the names of instances, and then matches with similar questions from communitycontributed QA websites as answers. The demonstration is conducted on a million-scale dataset of Web images and QA pairs in the domain of fashion products. Asking a multimedia question through FashionAsk can take as short as five …

Community As A Connector: Associating Faces With Celebrity Names In Web Videos, Zhineng Chen, Chong-Wah Ngo, Juan Cao, Wei Zhang

Research Collection School Of Computing and Information Systems

Associating celebrity faces appearing in videos with their names is of increasingly importance with the popularity of both celebrity videos and related queries. However, the problem is not yet seriously studied in Web video domain. This paper proposes a Community connected Celebrity Name-Face Association approach (CCNFA), where the community is regarded as an intermediate connector to facilitate the association. Specifically, with the names and faces extracted from Web videos, C-CNFA decomposes the association task into a three-step framework: community discovering, community matching and celebrity face tagging. To achieve the goal of efficient name-face association under this umbrella, algorithms such as …

Video Hyperlinking: Libraries And Tools For Threading And Visualizing Large Video Collection, Lei Pang, Wei Zhang, Hung-Khoon Tan, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

While HTML documents could be effortlessly hyperlinked by markup tags, creation of the hyperlinks for multimedia objects is by no means easy due to the involvement of various visual processing units and intensive computational overhead. This paper introduces an open source, named VIREO-VH, which provides end-to-end support for creating hyperlinks to thread and visualize collections of videos. The software components include video pre-processing, bag-ofwords based inverted file indexing for scalable near-duplicate keyframe search, localization of partial near-duplicate segments, and galaxy visualization of video collection. The open source has been internally used by VIREO research team since 2007, and was evolved …

Vireo@Trecvid 2012: Searching With Topology, Recounting Will Small Concepts, Learning With Free Examples, Wei Zhang, Chun-Chet Tan, Shi-Ai Zhu, Ting Yao, Lei Pang, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

The vireo group participated in four tasks: instance search, multimedia event recounting, multimedia event detection, and semantic indexing. In this paper, we will present our approaches and discuss the evaluation results.

A Theoretical Framework For Serious Game Design: Exploring Pedagogy, Play And Fidelity And Their Implications For The Design Process, Pauline Rooney

Journal Articles

It is widely acknowledged that digital games can provide an engaging, motivating and “fun” experience for students. However an entertaining game does not necessarily constitute a meaningful, valuable learning experience. For this reason, experts espouse the importance of underpinning serious games with a sound theoretical framework which integrates and balances theories from two fields of practice: pedagogy and game design (Kiili, 2005; Seeney & Routledge, 2009). Additionally, with the advent of sophisticated, immersive technologies, and increasing interest in the opportunities for constructivist learning offered by these technologies, concepts of fidelity and its impact on student learning and engagement, have emerged …

3dme: 3d Media Express From Rgb-D Images, Tam Nguyen, Lusong Li, Jun Tan, Shuicheng Yan

Computer Science Faculty Publications

Considering the continuously increasing availability and accessibility of 3D media and the depth camera such as Kinect, we demonstrate an innovative 3D media system called 3DME. The objective of this demo is three-fold.

First, the demo exhibits the creation of 3D images from RGB-D images. Second, 3DME allows a user to insert impressive effects to the produced 3D content. Last but not least, our demo is one of the first attempts toward advertising with 3D content.

Hi, Magic Closet, Tell Me What To Wear!, Si Liu, Tam Nguyen, Jiashi Feng, Meng Wang, Shuicheng Yan

Computer Science Faculty Publications

In this demo, we present a practical system, "magic closet," for automatic occasion-oriented clothing pairing. Given a user-input occasion, e.g., wedding or shopping, the magic closet intelligently and automatically pairs the user-specified reference clothing (upper body or lower body) with the most suitable one from online shops. Two key criteria are explicitly considered for the magic closet system. One criterion is to dress properly, e.g., compared to suit pants, it is more decent to wear a cocktail dress for a banquet occasion. The other criterion is to dress aesthetically, e.g., a red T-shirt matches better with white pants than with …

Sense Beauty Via Face, Dressing, And/Or Voice, Tam Nguyen, Si Liu, Bingbing Ni, Jun Tan, Yong Rui, Shuicheng Yan

Computer Science Faculty Publications

Discovering the secret of beauty has been the pursuit of artists and philosophers for centuries. Nowadays, the computational model for beauty estimation has been actively explored in computer science community, yet with the focus mainly on facial features. In this work, we perform a comprehensive study of female attractiveness conveyed by single/multiple modalities of cues, i.e., face, dressing, and/or voice, and aim to uncover how different modalities individually and collectively affect the human sense of beauty. To this end, we collect the first Multi-Modality Beauty (M²B) dataset in the world for female attractiveness study, which is thoroughly annotated …

The Shanghai-Hongkong Team At Mediaeval2012: Violent Scene Detection Using Trajectory-Based Features, Yu-Gang Jiang, Qi Dai, Chun Chet Tan, Xiangyang Xue, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

The Violent Scene Detection task offers a very practical challenge in detecting complex and diverse violent video clips in movies. In this working note paper, we will briefly describe our system and discuss the results, which achieved top performance in mAP@201 and runner-up in mAP@100, among all 35 submissions worldwide. The central component of our system is a set of features derived from the appearance and motion of local patch trajectories [2]. We use these features and SVM classifier as the baseline approach and add in a few other components to further improve the performance. Our findings indicate that the …

Eating Alone, Together: New Forms Of Commensality, Catherine Grevet, Anthony Tang, Elizabeth Mynatt

Research Collection School Of Computing and Information Systems

Eating with others, or commensality, is an enjoyable activity that serves many important social functions; however, many individuals eat meals alone due to life circumstances, meaning that they miss out on these social benefits. We developed and deployed a simple technology probe providing social awareness around mealtimes to explore how social systems might help alleviate the loneliness of solitary dining. Our findings suggest that these systems can convey a sense of connectedness around a meal; further, our analysis revealed three themes relevant to systems of this type: that contextually-located peripheral awareness engenders connectedness; that such tools can foster a feeling …

Trajectory-Based Modeling Of Human Actions With Motion Reference Points, Yu-Gang Jiang, Qi Dai, Xiangyang Xue, Wei Liu, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Human action recognition in videos is a challenging problem with wide applications. State-of-the-art approaches often adopt the popular bag-of-features representation based on isolated local patches or temporal patch trajectories, where motion patterns like object relationships are mostly discarded. This paper proposes a simple representation specifically aimed at the modeling of such motion relationships. We adopt global and local reference points to characterize motion information, so that the final representation can be robust to camera movement. Our approach operates on top of visual codewords derived from local patch trajectories, and therefore does not require accurate foreground-background separation, which is typically a …

Understanding Engagement In Educational Computer Games, Fiona Fui-Hoon Nah, Yunjie Zhou, Adeline Boey, Hanji Li

Research Collection School Of Computing and Information Systems

This paper presents an empirical study to understand engagement in educational computer games. Engagement is defined as an experience that occupies an individual’s attention and captures one’s interest. The nature of engagement is viewed as comprising conditions, actions, and outcomes of engagement. The data collection method includes in-depth interviews with 12 educational computer game players who have experienced engagement in playing these games. We used the Grounded Theory (GT) approach to develop an understanding of user engagement in the computer game-based learning context.

Extending A Mobile Device's Interaction Space Through Body-Centric Interaction, Xiang Chen, Nicolai Marquardt, Anthony Tang, Sebastian Boring, Saul Greenberg

Research Collection School Of Computing and Information Systems

Modern mobile devices rely on the screen as a primary input modality. Yet the small screen real-estate limits interaction possibilities, motivating researchers to explore alternate input techniques. Within this arena, our goal is to develop Body-Centric Interaction with Mobile Devices: a class of input techniques that allow a person to position and orient her mobile device to navigate and manipulate digital content anchored in the space on and around the body. To achieve this goal, we explore such interaction in a bottom-up path of prototypes and implementations. From our experiences, as well as by examining related work, we discuss and …

The Fat Thumb: Using The Thumb's Contact Size For Single-Handed Mobile Interaction, Sebastian Boring, David Ledo, Xiang ‘Anthony’ Chen, Anthony Tang, Anthony Tang, Saul Greenberg

Research Collection School Of Computing and Information Systems

Modern mobile devices allow a rich set of multi-finger interactions that combine modes into a single fluid act. Such gestures may require the use of both hands: one holding the device while the other is interacting. While on the go, however, only one hand may be available to both hold the device and interact with it. In this demo, we present the Fat Thumb interaction technique, which uses the thumb's contact size as a form of simulated pressure. We present how this can be used, for example, to integrate panning and zooming into a single interaction. Contact size determines the …

Eliciting A Sensemaking Process From Verbal Protocols Of Reverse Engineers, Adam R. Bryant, Robert F. Mills, Gilbert L. Peterson, Michael R. Grimaila

Faculty Publications

A process of sensemaking in reverse engineering was elicited from verbal protocols of reverse engineers as they investigated the assembly code of executable programs. Four participants were observed during task performance and verbal protocols were collected and analyzed from two of the participants to determine their problem-solving states and characterize likely transitions between those states. From this analysis, a high-level process of sensemaking is described which represents hypothesis generation and information-seeking behaviors in reverse engineering within a framework of goal-directed planning. Future work in validation and application of the process is discussed.

Sampling And Ontologically Pooling Web Images For Visual Concept Learning, Shiai Zhu, Chong-Wah Ngo, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Sufficient training examples are essential for effective learning of semantic visual concepts. In practice, however, acquiring noise-free training examples has always been expensive. Recently the rapid popularization of social media websites, such as Flickr, has made it possible to collect training exemplars without human assistance. This paper proposes a novel and efficient approach to collect training samples from the noisily tagged Web images for visual concept learning, where we try to maximize two important criteria, relevancy and coverage, of the automatically generated training sets. For the former, a simple method named semantic field is introduced to handle the imprecise and …

User Assisted Tree Reconstruction From Point Clouds, William P. Leavenworth Ii

Department of Computer Graphics Technology Degree Theses

LIDAR is a useful tool for quickly digitalizing real world objects, but it usually takes some effort to produce a recognizable object from the raw input. When the object is a tree, the challenge is to create a three‐dimensional model that represents its general shape, while avoiding the influence of undersampling and noise. In the method developed for this research, the user creates a sketch overlaying a display of the raw input data. Each node in the sketch creates an estimated slice of the tree skeleton at that point, and the slices are connected according to the connectivity of the …

The Utilization Of Context To Adapt The Interface Of A Computer, Ludovic Delaveau

Purdue Polytechnic Masters Theses

This research examines the possibility for a computer to adapt its interface and its settings to the current environment, through the notion of situations and corresponding profiles. These situations are group of context elements and tasks or activities performed in that operating conditions, such as the activities conducted at the workplace or the relaxation time at home. The profiles are set of configuration values for the computer, such as the network services. Adapting the computer serves the user as it takes care of putting his computer in the right mode to be efficient in the activity performed. In order to …

Real Time Stereo Vision Using Dynamic Programming On A Graphics Processing Unit (Gpu), Janet Marie Mendoza, Rodrigo Antonio Romero, Olac Fuentes

COURI Symposium Abstracts, Summer 2012

Stereo vision is an important research topic in computer vision. It attempts to emulate the human vision system by using two or more images of the same scene to estimate depth information in three dimensions. The development of this topic is of much importance to many applications that vary from medical to entertainment. Currently, many researchers are searching for a method that will increase the speed of the process and still obtain high-quality results. Dynamic programming, which is a method for solving complex problems by breaking them down into simpler sub problems, has been successfully applied in stereo vision. We …

The Effects Of Diagrams And Relational Complexity On User Performance In Conditional Probability Problems In A Non-Learning Context, Vincent J. Kellen

College of Computing and Digital Media Dissertations

Many disciplines in everyday life depend on improved performance in conditional probability problems. Most adults struggle with conditional probability problems and several prior studies have shown participant accuracy is less than 50%. This study examined user performance when aided with computer-generated Venn and Euler type diagrams in a non-learning context. Despite the prevalence of research into diagrams and extensive research into conditional probability problem solving, this study is one of the only studies to apply theories of working memory to predict user performance in conditional probability problems with diagrams. Following relational complexity theory, this study manipulated problem complexity in computer …