Intercept Graph: An Interactive Radial Visualization For Comparison Of State Changes,
2021
Singapore Management University
Intercept Graph: An Interactive Radial Visualization For Comparison Of State Changes, Shaolun Ruan, Yong Wang, Qiang Guan
Research Collection School Of Computing and Information Systems
State change comparison of multiple data items is often necessary in multiple application domains, such as medical science, financial engineering, sociology, biological science, etc. Slope graphs and grouped bar charts have been widely used to show a “before-and-after” story of different data states and indicate their changes. However, they visualize state changes as either slope or difference of bars, which has been proved less effective for quantitative comparison. Also, both visual designs suffer from visual clutter issues with an increasing number of data items. In this paper, we propose Intercept Graph, a novel visual design to facilitate effective interactive comparison …
Self-Supervised Multi-Class Pre-Training For Unsupervised Anomaly Detection And Segmentation In Medical Images,
2021
Singapore Management University
Self-Supervised Multi-Class Pre-Training For Unsupervised Anomaly Detection And Segmentation In Medical Images, Yu Tian, Fengbei Liu, Guansong Pang, Yuanhong Chen, Yuyuan Liu, Johan W. Verjans, Rajvinder Singh
Research Collection School Of Computing and Information Systems
Unsupervised anomaly detection (UAD) that requires only normal (healthy) training images is an important tool for enabling the development of medical image analysis (MIA) applications, such as disease screening, since it is often difficult to collect and annotate abnormal (or disease) images in MIA. However, heavily relying on the normal images may cause the model training to overfit the normal class. Self-supervised pre-training is an effective solution to this problem. Unfortunately, current self-supervision methods adapted from computer vision are sub-optimal for MIA applications because they do not explore MIA domain knowledge for designing the pretext tasks or the training process. …
Wav-Bert: Cooperative Acoustic And Linguistic Representation Learning For Low-Resource Speech Recognition,
2021
Singapore Management University
Wav-Bert: Cooperative Acoustic And Linguistic Representation Learning For Low-Resource Speech Recognition, Guolin Zheng, Yubei Xiao, Ke Gong, Pan Zhou, Xiaodan Liang, Liang Lin
Research Collection School Of Computing and Information Systems
Unifying acoustic and linguistic representation learning has become increasingly crucial to transfer the knowledge learned on the abundance of high-resource language data for low-resource speech recognition. Existing approaches simply cascade pre-trained acoustic and language models to learn the transfer from speech to text. However, how to solve the representation discrepancy of speech and text is unexplored, which hinders the utilization of acoustic and linguistic information. Moreover, previous works simply replace the embedding layer of the pre-trained language model with the acoustic features, which may cause the catastrophic forgetting problem. In this work, we introduce Wav-BERT, a cooperative acoustic and linguistic …
Exploiting Reasoning Chains For Multi-Hop Science Question Answering,
2021
Chinese University of Hong Kong
Exploiting Reasoning Chains For Multi-Hop Science Question Answering, Weiwen Xu, Yang Deng, Huihui Zhang, Deng Cai, Wai Lam
Research Collection School Of Computing and Information Systems
We propose a novel Chain Guided Retriever reader (CGR) framework to model the reasoning chain for multi-hop Science Question Answering. Our framework is capable of performing explainable reasoning without the need of any corpus-specific annotations, such as the ground-truth reasoning chain, or human annotated entity mentions. Specifically, we first generate reasoning chains from a semantic graph constructed by Abstract Meaning Representation of retrieved evidence facts. A Chain-aware loss, concerning both local and global chain information, is also designed to enable the generated chains to serve as distant supervision signals for training the retriever, where reinforcement learning is also adopted to …
Towards Balancing Vr Immersion And Bystander Awareness,
2021
Singapore Management University
Towards Balancing Vr Immersion And Bystander Awareness, Yoshiki Kudo, Anthony Tang, Kazuyuki Fujita, Isamu Endo, Kazuki Takashima, Yoshifumi Kitamura
Research Collection School Of Computing and Information Systems
Head-mounted displays (HMDs) increase immersion into virtual worlds. The problem is that this limits headset users' awareness of bystanders: headset users cannot attend to bystanders' presence and activities. We call this the HMD boundary. We explore how to make the HMD boundary permeable by comparing different ways of providing informal awareness cues to the headset user about bystanders. We adapted and implemented three visualization techniques (Avatar View, Radar and Presence++) that share bystanders' location and orientation with headset users. We conducted a hybrid user and simulation study with three different types of VR content (high, medium, low interactivity) with twenty …
Acoustic/Gravity Wave Phenomena In Wide-Field Imaging: From Data Analysis To A Modeling Framework For Observability In The Mlt Region And Beyond,
2021
Embry-Riddle Aeronautical University
Acoustic/Gravity Wave Phenomena In Wide-Field Imaging: From Data Analysis To A Modeling Framework For Observability In The Mlt Region And Beyond, Jaime Aguilar Guerrero
Doctoral Dissertations and Master's Theses
Acoustic waves, gravity waves, and larger-scale tidal and planetary waves are significant drivers of the atmosphere’s dynamics and of the local and global circulation that have direct and indirect impacts on our weather and climate. Their measurements and characterization are fundamental challenges in Aeronomy that require a wide range of instrumentation with distinct operational principles. Most measurements share the common features of integrating optical emissions or effects on radio waves through deep layers of the atmosphere. The geometry of these integrations create line-of-sight effects that must be understood, described, and accounted for to properly present the measured data in traditional …
Mapping E-Commerce Locally And Beyond: Citt K12 Special Investigation Project,
2021
California State University, Long Beach
Mapping E-Commerce Locally And Beyond: Citt K12 Special Investigation Project, Thomas O’Brien, Deanna Matsumoto
Mineta Transportation Institute
As all aspects of the American workplace become automated or digitally enhanced to some degree, K12 educators have an increasing responsibility to help their students acquire the technical skills necessary to organize and interpret information. Increasingly, this is done through Geographic Information Systems (GIS), especially in careers related to transportation and logistics. The Center for International Trade & Transportation (CITT) at CSU Long Beach has developed this K12 Special Investigation Project to introduce ArcGIS StoryMaps, an engaging, accessible and sophisticated web-based GIS application. The lessons center on e-commerce and its accompanying environmental and economic impact. Still, the activities can be …
Masked Face Analysis Via Multi-Task Deep Learning,
2021
University of Dayton
Masked Face Analysis Via Multi-Task Deep Learning, Vatsa S. Patel, Zhongliang Nie, Trung-Nghia Le, Tam Van Nguyen
Computer Science Faculty Publications
Face recognition with wearable items has been a challenging task in computer vision and involves the problem of identifying humans wearing a face mask. Masked face analysis via multi-task learning could effectively improve performance in many fields of face analysis. In this paper, we propose a unified framework for predicting the age, gender, and emotions of people wearing face masks. We first construct FGNET-MASK, a masked face dataset for the problem. Then, we propose a multi-task deep learning model to tackle the problem. In particular, the multi-task deep learning model takes the data as inputs and shares their weight to …
Aixfood'21: 3rd Workshop On Aixfood,
2021
Singapore Management University
Aixfood'21: 3rd Workshop On Aixfood, Ricardo Guerrero, Michael Spranger, Shuqiang Jiang, Chong-Wah Ngo
Research Collection School Of Computing and Information Systems
Food and cooking analysis present exciting research and application challenges for modern AI systems, particularly in the context of multimodal data such as images or video. A meal that appears in a food image is a product of a complex progression of cooking stages, often described in the accompanying textual recipe form. In the cooking process, individual ingredients change their physical properties, become combined with other food components, all to produce a final, yet highly variable, appearance of the meal. Recognizing food items or meals on a plate from images or videos, their physical properties such as the amount, nutritional …
Prediction Of Synthetic Lethal Interactions In Human Cancers Using Multi-View Graph Auto-Encoder,
2021
Singapore Management University
Prediction Of Synthetic Lethal Interactions In Human Cancers Using Multi-View Graph Auto-Encoder, Zhifeng Hao, Di Wu, Yuan Fang, Min Wu, Ruichu Cai, Xiaoli Li
Research Collection School Of Computing and Information Systems
Synthetic lethality (SL) is a very important concept for the development of targeted anticancer drugs. However, experimental methods for SL detection often suffer from various issues like high cost and low consistency across cell lines. Hence, computational methods for predicting novel SLs have recently emerged as complements for wet-lab experiments. In addition, SL data can be represented as a graph where nodes are genes and edges are the SL interactions. It is thus motivated to design advanced graph-based machine learning algorithms for SL prediction. In this paper, we propose a novel SL prediction method using Multi-view Graph Auto-Encoder (SLMGAE). We …
The Efficacy Of Collaborative Authoring Of Video Scene Descriptions,
2021
Singapore Management University
The Efficacy Of Collaborative Authoring Of Video Scene Descriptions, Rosiana Natalie, Jolene Kar Inn Loh, Huei Suen Tan, Joshua Shi-Hao Tseng, Ian Luke Yi-Ren Chan, Ebrima H. Jarjue, Hernisa Kacorri, Kotaro Hara
Research Collection School Of Computing and Information Systems
The majority of online video contents remain inaccessible to people with visual impairments due to the lack of audio descriptions to depict the video scenes. Content creators have traditionally relied on professionals to author audio descriptions, but their service is costly and not readily-available. We investigate the feasibility of creating more cost-effective audio descriptions that are also of high quality by involving novices. Specifically, we designed, developed, and evaluated ViScene, a web-based collaborative audio description authoring tool that enables a sighted novice author and a reviewer either sighted or blind to interact and contribute to scene descriptions (SDs)—text that can …
Visionary Caption: Improving The Accessibility Of Presentation Slides Through Highlighting Visualization,
2021
Singapore Management University
Visionary Caption: Improving The Accessibility Of Presentation Slides Through Highlighting Visualization, Carmen Ji Yan Yip, Jie Mi Chong, Sin Yee Kwek, Yong Wang, Kotaro Hara
Research Collection School Of Computing and Information Systems
Presentation slides are widely used in occasions such as academic talks and business meetings. Captions placed on slides support deaf and hard of hearing (DHH) people to understand spoken contents, but simultaneously comprehending and associating visual contents on slides and caption text could be challenging. In this paper, we design and develop a visualization technique to highlight and associate chart on a slide and numerical data in caption. We first conduct a small formative study with people with and without hearing impairments to assess the value of the visualization technique using a lo-fidelity video prototype. We then develop Visionary Caption, …
Conquer: Contextual Query-Aware Ranking For Video Corpus Moment Retrieval,
2021
Singapore Management University
Conquer: Contextual Query-Aware Ranking For Video Corpus Moment Retrieval, Zhijian Hou, Chong-Wah Ngo, W. K. Chan
Research Collection School Of Computing and Information Systems
This paper tackles a recently proposed Video Corpus Moment Retrieval task. This task is essential because advanced video retrieval applications should enable users to retrieve a precise moment from a large video corpus. We propose a novel CONtextual QUery-awarE Ranking (CONQUER) model for effective moment localization and ranking. CONQUER explores query context for multi-modal fusion and representation learning in two different steps. The first step derives fusion weights for the adaptive combination of multi-modal video content. The second step performs bi-directional attention to tightly couple video and query as a single joint representation for moment localization. As query context is …
Visilence: An Interactive Visualization Tool For Error Resilience Analysis,
2021
Kent State University
Visilence: An Interactive Visualization Tool For Error Resilience Analysis, Shaolun Ruan, Yong Wang, Qiang Guan
Research Collection School Of Computing and Information Systems
Soft errors have become one of the major concerns for HPC applications, as those errors can result in seriously corrupted outcomes, such as silent data corruptions (SDCs). Prior studies on error resilience have studied the robustness of HPC applications. However, it is still difficult for program developers to identify potential vulnerability to soft errors. In this paper, we present Visilence, a novel visualization tool to visually analyze error vulnerability based on the control-flow graph generated from HPC applications. Visilence efficiently visualizes the affected program states under injected errors and presents the visual analysis of the most vulnerable parts of an …
Condensing A Sequence To One Informative Frame For Video Recognition,
2021
Singapore Management University
Condensing A Sequence To One Informative Frame For Video Recognition, Qiu. Zhaofan, Ting Yao, Yan Shu, Chong-Wah Ngo, Tao Mei
Research Collection School Of Computing and Information Systems
Video is complex due to large variations in motion and rich content in fine-grained visual details. Abstracting useful information from such information-intensive media requires exhaustive computing resources. This paper studies a two-step alternative that first condenses the video sequence to an informative" frame" and then exploits off-the-shelf image recognition system on the synthetic frame. A valid question is how to define" useful information" and then distill it from a video sequence down to one synthetic frame. This paper presents a novel Informative Frame Synthesis (IFS) architecture that incorporates three objective tasks, ie, appearance reconstruction, video categorization, motion estimation, and two …
Constrained Contrastive Distribution Learning For Unsupervised Anomaly Detection And Localisation In Medical Images,
2021
University of Adelaide
Constrained Contrastive Distribution Learning For Unsupervised Anomaly Detection And Localisation In Medical Images, Yu Tian, Guansong Pang, Fengbei Liu, Yuanhong Chen, Seon Ho Shin, Johan W. Verjans, Rajvinder Singh
Research Collection School Of Computing and Information Systems
Unsupervised anomaly detection (UAD) learns one-class classifiers exclusively with normal (i.e., healthy) images to detect any abnormal (i.e., unhealthy) samples that do not conform to the expected normal patterns. UAD has two main advantages over its fully supervised counterpart. Firstly, it is able to directly leverage large datasets available from health screening programs that contain mostly normal image samples, avoiding the costly manual labelling of abnormal samples and the subsequent issues involved in training with extremely class-imbalanced data. Further, UAD approaches can potentially detect and localise any type of lesions that deviate from the normal patterns. One significant challenge faced …
Learning To Adversarially Blur Visual Object Tracking,
2021
Singapore Management University
Learning To Adversarially Blur Visual Object Tracking, Qing Guo, Ziyi Cheng, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yang Liu, Jianjun Zhao
Research Collection School Of Computing and Information Systems
Motion blur caused by the moving of the object or camera during the exposure can be a key challenge for visual object tracking, affecting tracking accuracy significantly. In this work, we explore the robustness of visual object trackers against motion blur from a new angle, i.e., adversarial blur attack (ABA). Our main objective is to online transfer input frames to their natural motion-blurred counterparts while misleading the state-of-the-art trackers during the tracking process. To this end, we first design the motion blur synthesizing method for visual tracking based on the generation principle of motion blur, considering the motion information and …
Causal Attention For Unbiased Visual Recognition,
2021
Singapore Management University
Causal Attention For Unbiased Visual Recognition, Tan Wang, Chang Zhou, Qianru Sun, Hanwang Zhang
Research Collection School Of Computing and Information Systems
Attention module does not always help deep models learn causal features that are robust in any confounding context, e.g., a foreground object feature is invariant to different backgrounds. This is because the confounders trick the attention to capture spurious correlations that benefit the prediction when the training and testing data are IID (identical & independent distribution); while harm the prediction when the data are OOD (out-of-distribution). The sole fundamental solution to learn causal attention is by causal intervention, which requires additional annotations of the confounders, e.g., a “dog” model is learned within “grass+dog” and “road+dog” respectively, so the “grass” and …
Transporting Causal Mechanisms For Unsupervised Domain Adaptation,
2021
Singapore Management University
Transporting Causal Mechanisms For Unsupervised Domain Adaptation, Zhongqi Yue, Qianru Sun, Xian-Sheng Hua, Hanwang Zhang
Research Collection School Of Computing and Information Systems
Existing Unsupervised Domain Adaptation (UDA) literature adopts the covariate shift and conditional shift assumptions, which essentially encourage models to learn common features across domains. However, due to the lack of supervision in the target domain, they suffer from the semantic loss: the feature will inevitably lose nondiscriminative semantics in source domain, which is however discriminative in target domain. We use a causal view—transportability theory [41]—to identify that such loss is in fact a confounding effect, which can only be removed by causal intervention. However, the theoretical solution provided by transportability is far from practical for UDA, because it requires the …
Self-Regulation For Semantic Segmentation,
2021
Singapore Management University
Self-Regulation For Semantic Segmentation, Dong Zhang, Hanwang Zhang, Jinhui Tang, Xian-Sheng Hua, Qianru Sun
Research Collection School Of Computing and Information Systems
In this paper, we seek reasons for the two major failure cases in Semantic Segmentation (SS): 1) missing small objects or minor object parts, and 2) mislabeling minor parts of large objects as wrong classes. We have an interesting finding that Failure-1 is due to the underuse of detailed features and Failure-2 is due to the underuse of visual contexts. To help the model learn a better trade-off, we introduce several Self-Regulation (SR) losses for training SS neural networks. By “self”, we mean that the losses are from the model per se without using any additional data or supervision. By …
