Graphics and Human Computer Interfaces Commons^™

Electronic Device, Display And Touch-Sensitive User Interface (Nov 2012), Michael E. Miller, Jerald J. Muszak, Michael J. Telek

AFIT Patents

Display devices and methods for operating the same are provided. In one embodiment, the display device has an electronic display having an active area for presenting visual content; a housing holding the electronic display and having an opening allowing a person to view a first portion of the active area; and a bezel about the opening, the bezel covering a second portion of the active area and providing a window through which at least a part of the second portion can be viewed. A sensor system senses when a person is close to touching the bezel or when a person …

Cross-View Graph Embedding, Zhiwu Huang, S. Shan, H. Zhang, S. Lao, X. Chen

Research Collection School Of Computing and Information Systems

Recently, more and more approaches are emerging to solve the cross-view matching problem where reference samples and query samples are from different views. In this paper, inspired by Graph Embedding, we propose a unified framework for these cross-view methods called Cross-view Graph Embedding. The proposed framework can not only reformulate most traditional cross-view methods (e.g., CCA, PLS and CDFE), but also extend the typical single-view algorithms (e.g., PCA, LDA and LPP) to cross-view editions. Furthermore, our general framework also facilitates the development of new cross-view methods. In this paper, we present a new algorithm named Cross-view Local Discriminant Analysis (CLODA) …

Snap-And-Ask: Answering Multimodal Question By Naming Visual Instance, Wei Zhang, Lei Pang, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

In real-life, it is easier to provide a visual cue when asking a question about a possibly unfamiliar topic, for example, asking the question, “Where was this crop circle found?”. Providing an image of the instance is far more convenient than texting a verbose description of the visual properties, especially when the name of the query instance is not known. Nevertheless, having to identify the visual instance before processing the question and eventually returning the answer makes multimodal question-answering technically challenging. This paper addresses the problem of visual-totext naming through the paradigm of answering-by-search in a two-stage computational framework, which …

Video Hyperlinking: Libraries And Tools For Threading And Visualizing Large Video Collection, Lei Pang, Wei Zhang, Hung-Khoon Tan, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

While HTML documents could be effortlessly hyperlinked by markup tags, creation of the hyperlinks for multimedia objects is by no means easy due to the involvement of various visual processing units and intensive computational overhead. This paper introduces an open source, named VIREO-VH, which provides end-to-end support for creating hyperlinks to thread and visualize collections of videos. The software components include video pre-processing, bag-ofwords based inverted file indexing for scalable near-duplicate keyframe search, localization of partial near-duplicate segments, and galaxy visualization of video collection. The open source has been internally used by VIREO research team since 2007, and was evolved …

Fashionask: Pushing Community Answers To Your Fingertips, Wei Zhang, Lei Pang, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

We demonstrate a multimedia-based question-answering system, named FashionAsk, by allowing users to ask questions referring to pictures snapped by mobile devices. Specifically, instead of asking verbose questions to depict visual instances, direct pictures are provided as part of questions. To answer these multi-modal questions, FashionAsk performs a large-scale instance search to infer the names of instances, and then matches with similar questions from communitycontributed QA websites as answers. The demonstration is conducted on a million-scale dataset of Web images and QA pairs in the domain of fashion products. Asking a multimedia question through FashionAsk can take as short as five …

Predicting Domain Adaptivity: Redo Or Recycle?, Ting Yao, Chong-Wah Ngo, Shiai Zhu

Research Collection School Of Computing and Information Systems

Over the years, the academic researchers have contributed various visual concept classifiers. Nevertheless, given a new dataset, most researchers still prefer to develop large number of classifiers from scratch despite expensive labeling efforts and limited computing resources. A valid question is why not multimedia community “embrace the green” and recycle off-the-shelf classifiers for new dataset. The difficulty originates from the domain gap that there are many different factors that govern the development of a classifier and eventually drive its performance to emphasize certain aspects of dataset. Reapplying a classifier to an unseen dataset may end up GIGO (garbage in, garbage …

Community As A Connector: Associating Faces With Celebrity Names In Web Videos, Zhineng Chen, Chong-Wah Ngo, Juan Cao, Wei Zhang

Research Collection School Of Computing and Information Systems

Associating celebrity faces appearing in videos with their names is of increasingly importance with the popularity of both celebrity videos and related queries. However, the problem is not yet seriously studied in Web video domain. This paper proposes a Community connected Celebrity Name-Face Association approach (CCNFA), where the community is regarded as an intermediate connector to facilitate the association. Specifically, with the names and faces extracted from Web videos, C-CNFA decomposes the association task into a three-step framework: community discovering, community matching and celebrity face tagging. To achieve the goal of efficient name-face association under this umbrella, algorithms such as …

Vireo@Trecvid 2012: Searching With Topology, Recounting Will Small Concepts, Learning With Free Examples, Wei Zhang, Chun-Chet Tan, Shi-Ai Zhu, Ting Yao, Lei Pang, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

The vireo group participated in four tasks: instance search, multimedia event recounting, multimedia event detection, and semantic indexing. In this paper, we will present our approaches and discuss the evaluation results.

A Theoretical Framework For Serious Game Design: Exploring Pedagogy, Play And Fidelity And Their Implications For The Design Process, Pauline Rooney

Journal Articles

It is widely acknowledged that digital games can provide an engaging, motivating and “fun” experience for students. However an entertaining game does not necessarily constitute a meaningful, valuable learning experience. For this reason, experts espouse the importance of underpinning serious games with a sound theoretical framework which integrates and balances theories from two fields of practice: pedagogy and game design (Kiili, 2005; Seeney & Routledge, 2009). Additionally, with the advent of sophisticated, immersive technologies, and increasing interest in the opportunities for constructivist learning offered by these technologies, concepts of fidelity and its impact on student learning and engagement, have emerged …

3dme: 3d Media Express From Rgb-D Images, Tam Nguyen, Lusong Li, Jun Tan, Shuicheng Yan

Computer Science Faculty Publications

Considering the continuously increasing availability and accessibility of 3D media and the depth camera such as Kinect, we demonstrate an innovative 3D media system called 3DME. The objective of this demo is three-fold.

First, the demo exhibits the creation of 3D images from RGB-D images. Second, 3DME allows a user to insert impressive effects to the produced 3D content. Last but not least, our demo is one of the first attempts toward advertising with 3D content.

Hi, Magic Closet, Tell Me What To Wear!, Si Liu, Tam Nguyen, Jiashi Feng, Meng Wang, Shuicheng Yan

Computer Science Faculty Publications

In this demo, we present a practical system, "magic closet," for automatic occasion-oriented clothing pairing. Given a user-input occasion, e.g., wedding or shopping, the magic closet intelligently and automatically pairs the user-specified reference clothing (upper body or lower body) with the most suitable one from online shops. Two key criteria are explicitly considered for the magic closet system. One criterion is to dress properly, e.g., compared to suit pants, it is more decent to wear a cocktail dress for a banquet occasion. The other criterion is to dress aesthetically, e.g., a red T-shirt matches better with white pants than with …

Sense Beauty Via Face, Dressing, And/Or Voice, Tam Nguyen, Si Liu, Bingbing Ni, Jun Tan, Yong Rui, Shuicheng Yan

Computer Science Faculty Publications

Discovering the secret of beauty has been the pursuit of artists and philosophers for centuries. Nowadays, the computational model for beauty estimation has been actively explored in computer science community, yet with the focus mainly on facial features. In this work, we perform a comprehensive study of female attractiveness conveyed by single/multiple modalities of cues, i.e., face, dressing, and/or voice, and aim to uncover how different modalities individually and collectively affect the human sense of beauty. To this end, we collect the first Multi-Modality Beauty (M²B) dataset in the world for female attractiveness study, which is thoroughly annotated …

The Shanghai-Hongkong Team At Mediaeval2012: Violent Scene Detection Using Trajectory-Based Features, Yu-Gang Jiang, Qi Dai, Chun Chet Tan, Xiangyang Xue, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

The Violent Scene Detection task offers a very practical challenge in detecting complex and diverse violent video clips in movies. In this working note paper, we will briefly describe our system and discuss the results, which achieved top performance in mAP@201 and runner-up in mAP@100, among all 35 submissions worldwide. The central component of our system is a set of features derived from the appearance and motion of local patch trajectories [2]. We use these features and SVM classifier as the baseline approach and add in a few other components to further improve the performance. Our findings indicate that the …

Trajectory-Based Modeling Of Human Actions With Motion Reference Points, Yu-Gang Jiang, Qi Dai, Xiangyang Xue, Wei Liu, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Human action recognition in videos is a challenging problem with wide applications. State-of-the-art approaches often adopt the popular bag-of-features representation based on isolated local patches or temporal patch trajectories, where motion patterns like object relationships are mostly discarded. This paper proposes a simple representation specifically aimed at the modeling of such motion relationships. We adopt global and local reference points to characterize motion information, so that the final representation can be robust to camera movement. Our approach operates on top of visual codewords derived from local patch trajectories, and therefore does not require accurate foreground-background separation, which is typically a …

Eating Alone, Together: New Forms Of Commensality, Catherine Grevet, Anthony Tang, Elizabeth Mynatt

Research Collection School Of Computing and Information Systems

Eating with others, or commensality, is an enjoyable activity that serves many important social functions; however, many individuals eat meals alone due to life circumstances, meaning that they miss out on these social benefits. We developed and deployed a simple technology probe providing social awareness around mealtimes to explore how social systems might help alleviate the loneliness of solitary dining. Our findings suggest that these systems can convey a sense of connectedness around a meal; further, our analysis revealed three themes relevant to systems of this type: that contextually-located peripheral awareness engenders connectedness; that such tools can foster a feeling …

Understanding Engagement In Educational Computer Games, Fiona Fui-Hoon Nah, Yunjie Zhou, Adeline Boey, Hanji Li

Research Collection School Of Computing and Information Systems

This paper presents an empirical study to understand engagement in educational computer games. Engagement is defined as an experience that occupies an individual’s attention and captures one’s interest. The nature of engagement is viewed as comprising conditions, actions, and outcomes of engagement. The data collection method includes in-depth interviews with 12 educational computer game players who have experienced engagement in playing these games. We used the Grounded Theory (GT) approach to develop an understanding of user engagement in the computer game-based learning context.

The Fat Thumb: Using The Thumb's Contact Size For Single-Handed Mobile Interaction, Sebastian Boring, David Ledo, Xiang ‘Anthony’ Chen, Anthony Tang, Anthony Tang, Saul Greenberg

Research Collection School Of Computing and Information Systems

Modern mobile devices allow a rich set of multi-finger interactions that combine modes into a single fluid act. Such gestures may require the use of both hands: one holding the device while the other is interacting. While on the go, however, only one hand may be available to both hold the device and interact with it. In this demo, we present the Fat Thumb interaction technique, which uses the thumb's contact size as a form of simulated pressure. We present how this can be used, for example, to integrate panning and zooming into a single interaction. Contact size determines the …

Extending A Mobile Device's Interaction Space Through Body-Centric Interaction, Xiang Chen, Nicolai Marquardt, Anthony Tang, Sebastian Boring, Saul Greenberg

Research Collection School Of Computing and Information Systems

Modern mobile devices rely on the screen as a primary input modality. Yet the small screen real-estate limits interaction possibilities, motivating researchers to explore alternate input techniques. Within this arena, our goal is to develop Body-Centric Interaction with Mobile Devices: a class of input techniques that allow a person to position and orient her mobile device to navigate and manipulate digital content anchored in the space on and around the body. To achieve this goal, we explore such interaction in a bottom-up path of prototypes and implementations. From our experiences, as well as by examining related work, we discuss and …

Sampling And Ontologically Pooling Web Images For Visual Concept Learning, Shiai Zhu, Chong-Wah Ngo, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Sufficient training examples are essential for effective learning of semantic visual concepts. In practice, however, acquiring noise-free training examples has always been expensive. Recently the rapid popularization of social media websites, such as Flickr, has made it possible to collect training exemplars without human assistance. This paper proposes a novel and efficient approach to collect training samples from the noisily tagged Web images for visual concept learning, where we try to maximize two important criteria, relevancy and coverage, of the automatically generated training sets. For the former, a simple method named semantic field is introduced to handle the imprecise and …

Eliciting A Sensemaking Process From Verbal Protocols Of Reverse Engineers, Adam R. Bryant, Robert F. Mills, Gilbert L. Peterson, Michael R. Grimaila

Faculty Publications

A process of sensemaking in reverse engineering was elicited from verbal protocols of reverse engineers as they investigated the assembly code of executable programs. Four participants were observed during task performance and verbal protocols were collected and analyzed from two of the participants to determine their problem-solving states and characterize likely transitions between those states. From this analysis, a high-level process of sensemaking is described which represents hypothesis generation and information-seeking behaviors in reverse engineering within a framework of goal-directed planning. Future work in validation and application of the process is discussed.