Graphics and Human Computer Interfaces Commons^™

Vireo-Eurecom @ Trecvid 2019: Ad-Hoc Video Search (Avs), Phuong Anh Nguyen, Jiaxin Wu, Chong-Wah Ngo, Francis Danny, Benoit Huet

Research Collection School Of Computing and Information Systems

In this paper, we describe the systems developed for Ad-hoc Video Search (AVS) task at TRECVID 2019[1] and the achieved results.

Low-Resource Name Tagging Learned With Weakly Labeled Data, Yixin Cao, Zikun Hu, Tat-Seng Chua, Zhiyuan Liu, Heng Ji

Research Collection School Of Computing and Information Systems

Name tagging in low-resource languages or domains suffers from inadequate training data. Existing work heavily relies on additional information, while leaving those noisy annotations unexplored that extensively exist on the web. In this paper, we propose a novel neural model for name tagging solely based on weakly labeled (WL) data, so that it can be applied in any low-resource settings. To take the best advantage of all WL sentences, we split them into high-quality and noisy portions for two modules, respectively: (1) a classification module focusing on the large portion of noisy data can efficiently and robustly pretrain the tag …

Semi-Supervised Entity Alignment Via Joint Knowledge Embedding Model And Cross-Graph Model, Chengjiang Li, Yixin Cao, Lei Hou, Jiaxin Shi, Juanzi Li, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Entity alignment aims at integrating complementary knowledge graphs (KGs) from different sources or languages, which may benefit many knowledge-driven applications. It is challenging due to the heterogeneity of KGs and limited seed alignments. In this paper, we propose a semi-supervised entity alignment method by joint Knowledge Embedding model and Cross-Graph model (KECG). It can make better use of seed alignments to propagate over the entire graphs with KG-based constraints. Specifically, as for the knowledge embedding model, we utilize TransE to implicitly complete two KGs towards consistency and learn relational constraints between entities. As for the cross-graph model, we extend Graph …

Revisiting Collaboration Through Mixed Reality: The Evolution Of Groupware, Barrett Ens, Joel Lanir, Anthony Tang, Scott Bateman, Gun Lee, Thammathip Piumsomboon, Mark Billinghurst

Research Collection School Of Computing and Information Systems

Collaborative Mixed Reality (MR) systems are at a critical point in time as they are soon to become more commonplace. However, MR technology has only recently matured to the point where researchers can focus deeply on the nuances of supporting collaboration, rather than needing to focus on creating the enabling technology. In parallel, but largely independently, the field of Computer Supported Cooperative Work (CSCW) has focused on the fundamental concerns that underlie human communication and collaboration over the past 30-plus years. Since MR research is now on the brink of moving into the real world, we reflect on three decades …

Shellnet: Efficient Point Cloud Convolutional Neural Networks Using Concentric Shells Statistics, Zhiyuan Zhang, Binh-Son Hua, Sai-Kit Yeung

Research Collection School Of Computing and Information Systems

Deep learning with 3D data has progressed significantly since the introduction of convolutional neural networks that can handle point order ambiguity in point cloud data. While being able to achieve good accuracies in various scene understanding tasks, previous methods often have low training speed and complex network architecture. In this paper, we address these problems by proposing an efficient end-to-end permutation invariant convolution for point cloud deep learning. Our simple yet effective convolution operator named ShellConv uses statistics from concentric spherical shells to define representative features and resolve the point order ambiguity, allowing traditional convolution to perform on such features. …

Digital Addiction: A Conceptual Overview, Amarjit Kumar Singh, Pawan Kumar Singh

Library Philosophy and Practice (e-journal)

Abstract

Digital addiction referred to an impulse control disorder that involves the obsessive use of digital devices, digital technologies, and digital platforms, i.e. internet, video game, online platforms, mobile devices, digital gadgets, and social network platform. It is an emerging domain of Cyberpsychology (Singh, Amarjit Kumar and Pawan Kumar Singh; 2019), which explore a problematic usage of digital media, device, and platforms by being obsessive and excessive. This article analyses, reviewed the current research, and established a conceptual overview on the digital addiction. The research literature on digital addiction has proliferated. However, we tried to categories the digital addiction, according …

Vrsensory: Designing Inclusive Virtual Games With Neurodiverse Children, Ben Wasserman, Derek Prate, Bryce Purnell, Alex Muse, Kaitlyn Abdo, Kendra Day, Louanne Boyd

Engineering Faculty Articles and Research

We explore virtual environments and accompanying interaction styles to enable inclusive play. In designing games for three neurodiverse children, we explore how designing for sensory diversity can be understood through a formal game design framework. Our process reveals that by using sensory processing needs as requirements we can make sensory and social accessible play spaces. We contribute empirical findings for accommodating sensory differences for neurodiverse children in a way that supports inclusive play. Specifically, we detail the sensory driven design choices that not only support the enjoyability of the leisure activities, but that also support the social inclusion of sensory-diverse …

Reachnn: Reachability Analysis Of Neural-Network Controlled Systems, Chao Huang, Jiameng Fan, Wenchao Li, Xin Chen, Qi Zhu

Computer Science Faculty Publications

Applying neural networks as controllers in dynamical systems has shown great promises. However, it is critical yet challenging to verify the safety of such control systems with neural-network controllers in the loop. Previous methods for verifying neural network controlled systems are limited to a few specific activation functions. In this work, we propose a new reachability analysis approach based on Bernstein polynomials that can verify neural-network controlled systems with a more general form of activation functions, i.e., as long as they ensure that the neural networks are Lipschitz continuous. Specifically, we consider abstracting feedforward neural networks with Bernstein polynomials for …

Fusion Of Multimodal Embeddings For Ad-Hoc Video Search, Danny Francis, Phuong Anh Nguyen, Benoit Huet, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

The challenge of Ad-Hoc Video Search (AVS) originates from free-form (i.e., no pre-defined vocabulary) and freestyle (i.e., natural language) query description. Bridging the semantic gap between AVS queries and videos becomes highly difficult as evidenced from the low retrieval accuracy of AVS benchmarking in TRECVID. In this paper, we study a new method to fuse multimodal embeddings which have been derived based on completely disjoint datasets. This method is tested on two datasets for two distinct tasks: on MSR-VTT for unique video retrieval and on V3C1 for multiple videos retrieval.

Mixed-Dish Recognition With Contextual Relation Networks, Lixi Deng, Jingjing Chen, Qianru Sun, Xiangnan He, Sheng Tang, Zhaoyan Ming, Yongdong Zhang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Mixed dish is a food category that contains different dishes mixed in one plate, and is popular in Eastern and Southeast Asia. Recognizing individual dishes in a mixed dish image is important for health related applications, e.g. calculating the nutrition values. However, most existing methods that focus on single dish classification are not applicable to mixed-dish recognition. The new challenge in recognizing mixed-dish images are the complex ingredient combination and severe overlap among different dishes. In order to tackle these problems, we propose a novel approach called contextual relation networks (CR-Nets) that encodes the implicit and explicit contextual relations among …

Nonuniform Timeslicing Of Dynamic Graphs Based On Visual Complexity, Yong Wang, Daniel Archambault, Hammad Haleem, Torsten Moeller, Yanhong Wu, Huamin Qu

Research Collection School Of Computing and Information Systems

Uniform timeslicing of dynamic graphs has been used due to its convenience and uniformity across the time dimension. However, uniform timeslicing does not take the data set into account, which can generate cluttered timeslices with edge bursts and empty timeslices with few interactions. The graph mining filed has explored nonuniform timeslicing methods specifically designed to preserve graph features for mining tasks. In this paper, we propose a nonuni-form timeslicing approach for dynamic graph visualization. Our goal is to create timeslices of equal visual complexity. To this end, we adapt histogram equalization to create timeslices with a similar number of events, …

Multi-Label Thoracic Disease Image Classification With Cross-Attention Networks, Congbo Ma, Hu Wang, Steven C. H. Hoi

Research Collection School Of Computing and Information Systems

Automated disease classification of radiology images has been emerging as a promising technique to support clinical diagnosis and treatment planning. Unlike generic image classification tasks, a real-world radiology image classification task is significantly more challenging as it is far more expensive to collect the training data where the labeled data is in nature multi-label; and more seriously samples from easy classes often dominate; training data is highly class-imbalanced problem exists in practice as well. To overcome these challenges, in this paper, we propose a novel scheme of Cross-Attention Networks (CAN) for automated thoracic disease classification from chest x-ray images, which …

Semi-Supervised Deep Quantization For Cross-Modal Search, Xin Wang, Wenwu Zhu, Chenghao Liu

Research Collection School Of Computing and Information Systems

The problem of cross-modal similarity search, which aims at making efficient and accurate queries across multiple domains, has become a significant and important research topic. Composite quantization, a compact coding solution superior to hashing techniques, has shown its effectiveness for similarity search. However, most existing works utilizing composite quantization to search multi-domain content only consider either pairwise similarity information or class label information across different domains, which fails to tackle the semi-supervised problem in composite quantization. In this paper, we address the semi-supervised quantization problem by considering: (i) pairwise similarity information (without class label information) across different domains, which captures …

Rotation Invariant Convolutions For 3d Point Clouds Deep Learning, Zhiyuan Zhang, Binh-Son Hua, David W. Rosen, Sai-Kit Yeung

Research Collection School Of Computing and Information Systems

Recent progresses in 3D deep learning has shown that it is possible to design special convolution operators to consume point cloud data. However, a typical drawback is that rotation invariance is often not guaranteed, resulting in networks that generalizes poorly to arbitrary rotations. In this paper, we introduce a novel convolution operator for point clouds that achieves rotation invariance. Our core idea is to use low-level rotation invariant geometric features such as distances and angles to design a convolution operator for point cloud learning. The well-known point ordering problem is also addressed by a binning approach seamlessly built into the …

Anticipating Widespread Augmented Reality: Insights From The 2018 Ar Visioning Workshop, Gregory F. Welch, Gerd Bruder, Peter Squire, Ryan Schubert

Faculty Scholarship and Creative Works

In August of 2018 a group of academic, government, and industry experts in the field of Augmented Reality gathered for four days to consider potential technological and societal issues and opportunities that could accompany a future where AR is pervasive in location and duration of use. This report is intended to summarize some of the most novel and potentially impactful insights and opportunities identified by the group.

Our target audience includes AR researchers, government leaders, and thought leaders in general. It is our intent to share some compelling technological and societal questions that we believe are unique to AR, and …

A Multimodal Approach To Sarcasm Detection On Social Media, Dipto Das

Graduate Theses/Dissertations

In recent times, a major share of human communication takes place online. The main reason being the ease of communication on social networking sites (SNSs). Due to the variety and large number of users, SNSs have drawn the attention of the computer science (CS) community, particularly the affective computing (also known as emotional AI), information retrieval, natural language processing, and data mining groups. Researchers are trying to make computers understand the nuances of human communication including sentiment and sarcasm. Emotion or sentiment detection requires more insights about the communication than it does for factual information retrieval. Sarcasm detection is particularly …

Generalized Majorization-Minimization For Non-Convex Optimization, Hu Zhang, Pan Zhou, Yi Yang, Jiashi Feng

Research Collection School Of Computing and Information Systems

Majorization-Minimization (MM) algorithms optimize an objective function by iteratively minimizing its majorizing surrogate and offer attractively fast convergence rate for convex problems. However, their convergence behaviors for non-convex problems remain unclear. In this paper, we propose a novel MM surrogate function from strictly upper bounding the objective to bounding the objective in expectation. With this generalized surrogate conception, we develop a new optimization algorithm, termed SPI-MM, that leverages the recent proposed SPIDER for more efficient non-convex optimization. We prove that for finite-sum problems, the SPI-MM algorithm converges to an stationary point within deterministic and lower stochastic gradient complexity. To our …

Bidding Mechanisms In Graph Games, Guy Avni, Thomas A. Henzinger, Dorde Zikelic

Research Collection School Of Computing and Information Systems

In two-player games on graphs, the players move a token through a graph to produce a finite or infinite path, which determines the qualitative winner or quantitative payoff of the game. We study bidding games in which the players bid for the right to move the token. Several bidding rules were studied previously. In Richman bidding, in each round, the players simultaneously submit bids, and the higher bidder moves the token and pays the other player. Poorman bidding is similar except that the winner of the bidding pays the “bank” rather than the other player. Taxman bidding spans the spectrum …

Multimodal Transformer Networks For End-To-End Video-Grounded Dialogue Systems, Hung Le, Doyen Sahoo, Nancy F. Chen, Steven C. H. Hoi

Research Collection School Of Computing and Information Systems

Developing Video-Grounded Dialogue Systems (VGDS), where a dialogue is conducted based on visual and audio aspects of a given video, is significantly more challenging than traditional image or text-grounded dialogue systems because (1) feature space of videos span across multiple picture frames, making it difficult to obtain semantic information; and (2) a dialogue agent must perceive and process information from different modalities (audio, video, caption, etc.) to obtain a comprehensive understanding. Most existing work is based on RNNs and sequence-to-sequence architectures, which are not very effective for capturing complex long-term dependencies (like in videos). To overcome this, we propose Multimodal …

Kgat: Knowledge Graph Attention Network For Recommendation, Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

To provide more accurate, diverse, and explainable recommendation, it is compulsory to go beyond modeling user-item interactions and take side information into account. Traditional methods like factorization machine (FM) cast it as a supervised learning problem, which assumes each interaction as an independent instance with side information encoded. Due to the overlook of the relations among instances or items (e.g., the director of a movie is also an actor of another movie), these methods are insufficient to distill the collaborative signal from the collective behaviors of users. In this work, we investigate the utility of knowledge graph (KG), which breaks …