Open Access. Powered by Scholars. Published by Universities.®

Graphics and Human Computer Interfaces Commons™

Open Access. Powered by Scholars. Published by Universities.®

Singapore Management University

Discipline
Keyword
Publication Year
Publication
Publication Type

Articles 451 - 480 of 959

Full-Text Articles in Graphics and Human Computer Interfaces

Aixfood'21: 3rd Workshop On Aixfood, Ricardo Guerrero, Michael Spranger, Shuqiang Jiang, Chong-Wah Ngo Oct 2021

Aixfood'21: 3rd Workshop On Aixfood, Ricardo Guerrero, Michael Spranger, Shuqiang Jiang, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Food and cooking analysis present exciting research and application challenges for modern AI systems, particularly in the context of multimodal data such as images or video. A meal that appears in a food image is a product of a complex progression of cooking stages, often described in the accompanying textual recipe form. In the cooking process, individual ingredients change their physical properties, become combined with other food components, all to produce a final, yet highly variable, appearance of the meal. Recognizing food items or meals on a plate from images or videos, their physical properties such as the amount, nutritional …

Go to article

Visilence: An Interactive Visualization Tool For Error Resilience Analysis, Shaolun Ruan, Yong Wang, Qiang Guan Oct 2021

Visilence: An Interactive Visualization Tool For Error Resilience Analysis, Shaolun Ruan, Yong Wang, Qiang Guan

Research Collection School Of Computing and Information Systems

Soft errors have become one of the major concerns for HPC applications, as those errors can result in seriously corrupted outcomes, such as silent data corruptions (SDCs). Prior studies on error resilience have studied the robustness of HPC applications. However, it is still difficult for program developers to identify potential vulnerability to soft errors. In this paper, we present Visilence, a novel visualization tool to visually analyze error vulnerability based on the control-flow graph generated from HPC applications. Visilence efficiently visualizes the affected program states under injected errors and presents the visual analysis of the most vulnerable parts of an …

Go to article

Constrained Contrastive Distribution Learning For Unsupervised Anomaly Detection And Localisation In Medical Images, Yu Tian, Guansong Pang, Fengbei Liu, Yuanhong Chen, Seon Ho Shin, Johan W. Verjans, Rajvinder Singh Oct 2021

Constrained Contrastive Distribution Learning For Unsupervised Anomaly Detection And Localisation In Medical Images, Yu Tian, Guansong Pang, Fengbei Liu, Yuanhong Chen, Seon Ho Shin, Johan W. Verjans, Rajvinder Singh

Research Collection School Of Computing and Information Systems

Unsupervised anomaly detection (UAD) learns one-class classifiers exclusively with normal (i.e., healthy) images to detect any abnormal (i.e., unhealthy) samples that do not conform to the expected normal patterns. UAD has two main advantages over its fully supervised counterpart. Firstly, it is able to directly leverage large datasets available from health screening programs that contain mostly normal image samples, avoiding the costly manual labelling of abnormal samples and the subsequent issues involved in training with extremely class-imbalanced data. Further, UAD approaches can potentially detect and localise any type of lesions that deviate from the normal patterns. One significant challenge faced …

Go to article

Learning To Adversarially Blur Visual Object Tracking, Qing Guo, Ziyi Cheng, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yang Liu, Jianjun Zhao Oct 2021

Learning To Adversarially Blur Visual Object Tracking, Qing Guo, Ziyi Cheng, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yang Liu, Jianjun Zhao

Research Collection School Of Computing and Information Systems

Motion blur caused by the moving of the object or camera during the exposure can be a key challenge for visual object tracking, affecting tracking accuracy significantly. In this work, we explore the robustness of visual object trackers against motion blur from a new angle, i.e., adversarial blur attack (ABA). Our main objective is to online transfer input frames to their natural motion-blurred counterparts while misleading the state-of-the-art trackers during the tracking process. To this end, we first design the motion blur synthesizing method for visual tracking based on the generation principle of motion blur, considering the motion information and …

Go to article

Self-Regulation For Semantic Segmentation, Dong Zhang, Hanwang Zhang, Jinhui Tang, Xian-Sheng Hua, Qianru Sun Oct 2021

Self-Regulation For Semantic Segmentation, Dong Zhang, Hanwang Zhang, Jinhui Tang, Xian-Sheng Hua, Qianru Sun

Research Collection School Of Computing and Information Systems

In this paper, we seek reasons for the two major failure cases in Semantic Segmentation (SS): 1) missing small objects or minor object parts, and 2) mislabeling minor parts of large objects as wrong classes. We have an interesting finding that Failure-1 is due to the underuse of detailed features and Failure-2 is due to the underuse of visual contexts. To help the model learn a better trade-off, we introduce several Self-Regulation (SR) losses for training SS neural networks. By “self”, we mean that the losses are from the model per se without using any additional data or supervision. By …

Go to article

Differentiated Learning For Multi-Modal Domain Adaptation, Jianming Lv, Kaijie Liu, Shengfeng He Oct 2021

Differentiated Learning For Multi-Modal Domain Adaptation, Jianming Lv, Kaijie Liu, Shengfeng He

Research Collection School Of Computing and Information Systems

Directly deploying a trained multi-modal classifier to a new environment usually leads to poor performance due to the well-known domain shift problem. Existing multi-modal domain adaptation methods treated each modality equally and optimize the sub-models of different modalities synchronously. However, as observed in this paper, the degrees of domain shift in different modalities are usually diverse. We propose a novel Differentiated Learning framework to make use of the diversity between multiple modalities for more effective domain adaptation. Specifically, we model the classifiers of different modalities as a group of teacher/student sub-models, and a novel Prototype based Reliability Measurement is presented …

Go to article

Few-Shot Breast Cancer Metastases Classification Via Unsupervised Cell Ranking, Jiaojiao Chen, Jianbo Jiao, Shengfeng He, Guoqiang Han, Jing Qin Sep 2021

Few-Shot Breast Cancer Metastases Classification Via Unsupervised Cell Ranking, Jiaojiao Chen, Jianbo Jiao, Shengfeng He, Guoqiang Han, Jing Qin

Research Collection School Of Computing and Information Systems

Tumor metastases detection is of great importance for the treatment of breast cancer patients. Various CNN (convolutional neural network) based methods get excellent performance in object detection/ segmentation. However, the detection of metastases in hematoxylin and eosin (H&E) stained whole-slide images (WSI) is still challenging mainly due to two aspects. (1) The resolution of the image is too large. (2) lacking labeled training data. Whole-slide images generally stored in a multi-resolution structure with multiple downsampled tiles. It is difficult to feed the whole image into memory without compression. Moreover, labeling images for the pathologists are time-consuming and expensive. In this …

Go to article

Semi-Supervised Semantic Visualization For Networked Documents, Delvin Ce Zhang, Hady W. Lauw Sep 2021

Semi-Supervised Semantic Visualization For Networked Documents, Delvin Ce Zhang, Hady W. Lauw

Research Collection School Of Computing and Information Systems

Semantic interpretability and visual expressivity are important objectives in exploratory analysis of text. On the one hand, while some documents may have explicit categories, we could develop a better understanding of a corpus by studying its finer-grained structures, which may be latent. By inferring latent topics and discovering keywords associated with each topic, one obtains a semantic interpretation of the corpus. One the other hand, by visualizing documents, latent topics, and category labels on the same plot, one gains a bird’s eye view of the relationships among documents, topics, and various categories. Semantic visualization is a class of methods that …

Go to article

An Empirical Study Of The Discreteness Prior In Low-Rank Matrix Completion, Rodrigo Alves, Antoine Ledent, Renato Assunção, Marius And Kloft Aug 2021

An Empirical Study Of The Discreteness Prior In Low-Rank Matrix Completion, Rodrigo Alves, Antoine Ledent, Renato Assunção, Marius And Kloft

Research Collection School Of Computing and Information Systems

A reasonable assumption in recommender systems is that the rows (users) and columns (items) of the rating matrix can be split into groups (communities) with the following property: each entry of the matrix is the sum of components corresponding to community behavior and a purely low-rank component corresponding to individual behavior. We investigate (1) whether such a structure is present in real-world datasets, (2) whether the knowledge of the existence of such structure alone can improve performance, without explicit information about the community memberships. To these ends, we formulate a joint optimization problem over all (completed matrix, set of communities) …

Go to article

A Survey On Ml4vis: Applying Machine Learning Advances To Data Visualization, Qianwen Wang, Zhutian Chen, Yong Wang, Huamin Qu Aug 2021

A Survey On Ml4vis: Applying Machine Learning Advances To Data Visualization, Qianwen Wang, Zhutian Chen, Yong Wang, Huamin Qu

Research Collection School Of Computing and Information Systems

Inspired by the great success of machine learning (ML), researchers have applied ML techniques to visualizations to achieve a better design, development, and evaluation of visualizations. This branch of studies, known as ML4VIS, is gaining increasing research attention in recent years. To successfully adapt ML techniques for visualizations, a structured understanding of the integration of ML4VIS is needed. In this article, we systematically survey 88 ML4VIS studies, aiming to answer two motivating questions: “what visualization processes can be assisted by ML?” and “how ML techniques can be used to solve visualization problems? ” This survey reveals seven main processes where …

Go to article

Forecasting Interaction Order On Temporal Graphs, Wenwen Xia, Yuchen Li, Jianwei Tian, Shenghong Li Aug 2021

Forecasting Interaction Order On Temporal Graphs, Wenwen Xia, Yuchen Li, Jianwei Tian, Shenghong Li

Research Collection School Of Computing and Information Systems

Link prediction is a fundamental task for graph analysis and the topic has been studied extensively for static or dynamic graphs. Essentially, the link prediction is formulated as a binary classification problem about two nodes. However, for temporal graphs, links (or interactions) among node sets appear in sequential orders. And the orders may lead to interesting applications. While a binary link prediction formulation fails to handle such an order-sensitive case. In this paper, we focus on such an interaction order prediction (IOP) problem among a given node set on temporal graphs. For the technical aspect, we develop a graph neural …

Go to article

Bidding Mechanisms In Graph Games, Guy Avni, Thomas A. Henzinger, Dorde Zikelic Aug 2021

Bidding Mechanisms In Graph Games, Guy Avni, Thomas A. Henzinger, Dorde Zikelic

Research Collection School Of Computing and Information Systems

A graph game proceeds as follows: two players move a token through a graph to produce a finite or infinite path, which determines the payoff of the game. We study bidding games in which in each turn, an auction determines which player moves the token. Bidding games were largely studied in combination with two variants of first-price auctions called “Richman” and “poorman” bidding. We study taxman bidding, which span the spectrum between the two. The game is parameterized by a constant τ∈[0,1]: portion τ of the winning bid is paid to the other player, and portion 1−τ to the bank. …

Go to article

Towards Generative Aspect-Based Sentiment Analysis, Wenxuan Zhang, Xin Li, Yang Deng, Lidong Bing, Wai Lam Aug 2021

Towards Generative Aspect-Based Sentiment Analysis, Wenxuan Zhang, Xin Li, Yang Deng, Lidong Bing, Wai Lam

Research Collection School Of Computing and Information Systems

Aspect-based sentiment analysis (ABSA) has received increasing attention recently. Most existing work tackles ABSA in a discriminative manner, designing various task-specific classification networks for the prediction. Despite their effectiveness, these methods ignore the rich label semantics in ABSA problems and require extensive task-specific designs. In this paper, we propose to tackle various ABSA tasks in a unified generative framework. Two types of paradigms, namely annotation-style and extraction-style modeling, are designed to enable the training process by formulating each ABSA task as a text generation problem. We conduct experiments on four ABSA tasks across multiple benchmark datasets where our proposed generative …

Go to article

Invertible Grayscale With Sparsity Enforcing Priors, Yong Du, Yangyang Xu, Taizhong Ye, Qiang Wen, Chufeng Xiao, Junyu Dong, Guoqiang Han, Shengfeng He Aug 2021

Invertible Grayscale With Sparsity Enforcing Priors, Yong Du, Yangyang Xu, Taizhong Ye, Qiang Wen, Chufeng Xiao, Junyu Dong, Guoqiang Han, Shengfeng He

Research Collection School Of Computing and Information Systems

Color dimensionality reduction is believed as a non-invertible process, as re-colorization results in perceptually noticeable and unrecoverable distortion. In this article, we propose to convert a color image into a grayscale image that can fully recover its original colors, and more importantly, the encoded information is discriminative and sparse, which saves storage capacity. Particularly, we design an invertible deep neural network for color encoding and decoding purposes. This network learns to generate a residual image that encodes color information, and it is then combined with a base grayscale image for color recovering. In this way, the non-differentiable compression process (e.g., …

Go to article

Learning Interpretable Concept Groups In Cnns, Saurabh Varshneya, Antoine Ledent, Rob Vandermeulen, Yunwen Lei, Matthias Enders, Damian Borth, Marius Kloft Aug 2021

Learning Interpretable Concept Groups In Cnns, Saurabh Varshneya, Antoine Ledent, Rob Vandermeulen, Yunwen Lei, Matthias Enders, Damian Borth, Marius Kloft

Research Collection School Of Computing and Information Systems

We propose a novel training methodology---Concept Group Learning (CGL)---that encourages training of interpretable CNN filters by partitioning filters in each layer into concept groups, each of which is trained to learn a single visual concept. We achieve this through a novel regularization strategy that forces filters in the same group to be active in similar image regions for a given layer. We additionally use a regularizer to encourage a sparse weighting of the concept groups in each layer so that a few concept groups can have greater importance than others. We quantitatively evaluate CGL's model interpretability using standard interpretability evaluation …

Go to article

Fine-Grained Analysis Of Structured Output Prediction, Waleed Mustafa, Yunwen Lei, Antoine Ledent, Marius And Kloft Aug 2021

Fine-Grained Analysis Of Structured Output Prediction, Waleed Mustafa, Yunwen Lei, Antoine Ledent, Marius And Kloft

Research Collection School Of Computing and Information Systems

In machine learning we often encounter structured output prediction problems (SOPPs), i.e. problems where the output space admits a rich internal structure. Application domains where SOPPs naturally occur include natural language processing, speech recognition, and computer vision. Typical SOPPs have an extremely large label set, which grows exponentially as a function of the size of the output. Existing generalization analysis implies generalization bounds with at least a square-root dependency on the cardinality d of the label set, which can be vacuous in practice. In this paper, we significantly improve the state of the art by developing novel high-probability bounds with …

Go to article

Leveraging Two Types Of Global Graph For Sequential Fashion Recommendation, Yujuan Ding, Yunshan Ma, Wai Keung Wong, Tat‑Seng Chua Aug 2021

Leveraging Two Types Of Global Graph For Sequential Fashion Recommendation, Yujuan Ding, Yunshan Ma, Wai Keung Wong, Tat‑Seng Chua

Research Collection School Of Computing and Information Systems

Sequential fashion recommendation is of great significance in online fashion shopping, which accounts for an increasing portion of either fashion retailing or online e-commerce. The key to building an effective sequential fashion recommendation model lies in capturing two types of patterns: the personal fashion preference of users and the transitional relationships between adjacent items. The two types of patterns are usually related to user-item interaction and item-item transition modeling respectively. However, due to the large sets of users and items as well as the sparse historical interactions, it is difficult to train an effective and efficient sequential fashion recommendation model. …

Go to article

Are Missing Links Predictable? An Inferential Benchmark For Knowledge Graph Completion, Yixin Cao, Xiang Ji, Xin Lv, Juanzi Li, Yonggang Wen, Hanwang Zhang Aug 2021

Are Missing Links Predictable? An Inferential Benchmark For Knowledge Graph Completion, Yixin Cao, Xiang Ji, Xin Lv, Juanzi Li, Yonggang Wen, Hanwang Zhang

Research Collection School Of Computing and Information Systems

We present InferWiki, a Knowledge Graph Completion (KGC) dataset that improves upon existing benchmarks in inferential ability, assumptions, and patterns. First, each testing sample is predictable with supportive data in the training set. To ensure it, we propose to utilize rule-guided train/test generation, instead of conventional random split. Second, InferWiki initiates the evaluation following the open-world assumption and improves the inferential difficulty of the closed-world assumption, by providing manually annotated negative and unknown triples. Third, we include various inference patterns (e.g., reasoning path length and types) for comprehensive evaluation. In experiments, we curate two settings of InferWiki varying in sizes …

Go to article

Learning From Miscellaneous Other-Class Words For Few-Shot Named Entity Recognition, Meihan Tong, Shuai Wang, Bin Xu, Yixin Cao, Minghui Liu, Lei Hou, Juanzi Li Aug 2021

Learning From Miscellaneous Other-Class Words For Few-Shot Named Entity Recognition, Meihan Tong, Shuai Wang, Bin Xu, Yixin Cao, Minghui Liu, Lei Hou, Juanzi Li

Research Collection School Of Computing and Information Systems

Few-shot Named Entity Recognition (NER) exploits only a handful of annotations to identify and classify named entity mentions. Prototypical network shows superior performance on few-shot NER. However, existing prototypical methods fail to differentiate rich semantics in other-class words, which will aggravate overfitting under few shot scenario. To address the issue, we propose a novel model, Mining Undefined Classes from Other-class (MUCO), that can automatically induce different undefined classes from the other class to improve few-shot NER. With these extra-labeled undefined classes, our method will improve the discriminative ability of NER classifier and enhance the understanding of predefined classes with stand-by …

Go to article

How Knowledge Graph And Attention Help? A Qualitative Analysis Into Bag-Level Relation Extraction, Zikun Hu, Yixin Cao, Lifu Huang, Tat-Seng Chua Aug 2021

How Knowledge Graph And Attention Help? A Qualitative Analysis Into Bag-Level Relation Extraction, Zikun Hu, Yixin Cao, Lifu Huang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Knowledge Graph (KG) and attention mechanism have been demonstrated effective in introducing and selecting useful information for weakly supervised methods. However, only qualitative analysis and ablation study are provided as evidence. In this paper, we contribute a dataset and propose a paradigm to quantitatively evaluate the effect of attention and KG on bag-level relation extraction (RE). We find that (1) higher attention accuracy may lead to worse performance as it may harm the model’s ability to extract entity mention features; (2) the performance of attention is largely influenced by various noise distribution patterns, which is closely related to real-world datasets; …

Go to article

How Important Is The Train-Validation Split In Meta-Learning?, Yu Bai, Minshuo Chen, Pan Zhou, Tuo Zhao, D. Jason Lee, Sham Kakade, Huan Wang, Caiming Xiong Jul 2021

How Important Is The Train-Validation Split In Meta-Learning?, Yu Bai, Minshuo Chen, Pan Zhou, Tuo Zhao, D. Jason Lee, Sham Kakade, Huan Wang, Caiming Xiong

Research Collection School Of Computing and Information Systems

Meta-learning aims to perform fast adaptation on a new task through learning a “prior” from multiple existing tasks. A common practice in meta-learning is to perform a train-validation split (train-val method) where the prior adapts to the task on one split of the data, and the resulting predictor is evaluated on another split. Despite its prevalence, the importance of the train-validation split is not well understood either in theory or in practice, particularly in comparison to the more direct train-train method, which uses all the pertask data for both training and evaluation. We provide a detailed theoretical study on whether …

Go to article

Emotioncues: Emotion-Oriented Visual Summarization Of Classroom Videos, Haipeng Zeng, Xinhuan Shu, Yanbang Wang, Yong Wang, Liguo Zhang, Ting-Chuen Pong, Huamin Qu Jul 2021

Emotioncues: Emotion-Oriented Visual Summarization Of Classroom Videos, Haipeng Zeng, Xinhuan Shu, Yanbang Wang, Yong Wang, Liguo Zhang, Ting-Chuen Pong, Huamin Qu

Research Collection School Of Computing and Information Systems

Analyzing students' emotions from classroom videos can help both teachers and parents quickly know the engagement of students in class. The availability of high-definition cameras creates opportunities to record class scenes. However, watching videos is time-consuming, and it is challenging to gain a quick overview of the emotion distribution and find abnormal emotions. In this paper, we propose EmotionCues, a visual analytics system to easily analyze classroom videos from the perspective of emotion summary and detailed analysis, which integrates emotion recognition algorithms with visualizations. It consists of three coordinated views: a summary view depicting the overall emotions and their dynamic …

Go to article

Dehumor: Visual Analytics For Decomposing Humor, Xingbo Wang, Yao Ming, Tongshuang Wu, Haipeng Zeng, Yong Wang, Huamin Qu Jul 2021

Dehumor: Visual Analytics For Decomposing Humor, Xingbo Wang, Yao Ming, Tongshuang Wu, Haipeng Zeng, Yong Wang, Huamin Qu

Research Collection School Of Computing and Information Systems

Despite being a critical communication skill, grasping humor is challenginga successful use of humor requires a mixture of both engaging content build-up and an appropriate vocal delivery (e.g., pause). Prior studies on computational humor emphasize the textual and audio features immediately next to the punchline, yet overlooking longer-term context setup. Moreover, the theories are usually too abstract for understanding each concrete humor snippet. To fill in the gap, we develop DeHumor, a visual analytical system for analyzing humorous behaviors in public speaking. To intuitively reveal the building blocks of each concrete example, DeHumor decomposes each humorous video into multimodal features …

Go to article

Unified Conversational Recommendation Policy Learning Via Graph-Based Reinforcement Learning, Yang Deng, Yaliang Li, Fei Sun, Bolin Ding, Wai Lam Jul 2021

Unified Conversational Recommendation Policy Learning Via Graph-Based Reinforcement Learning, Yang Deng, Yaliang Li, Fei Sun, Bolin Ding, Wai Lam

Research Collection School Of Computing and Information Systems

Conversational recommender systems (CRS) enable the traditional recommender systems to explicitly acquire user preferences towards items and attributes through interactive conversations. Reinforcement learning (RL) is widely adopted to learn conversational recommendation policies to decide what attributes to ask, which items to recommend, and when to ask or recommend, at each conversation turn. However, existing methods mainly target at solving one or two of these three decision-making problems in CRS with separated conversation and recommendation components, which restrict the scalability and generality of CRS and fall short of preserving a stable training procedure. In the light of these challenges, we propose …

Go to article

Reciprocal Transformations For Unsupervised Video Object Segmentation, Sucheng Ren, Wenxi Liu, Yongtuo Liu, Haoxin Chen, Guoqiang Han, Shengfeng He Jun 2021

Reciprocal Transformations For Unsupervised Video Object Segmentation, Sucheng Ren, Wenxi Liu, Yongtuo Liu, Haoxin Chen, Guoqiang Han, Shengfeng He

Research Collection School Of Computing and Information Systems

Unsupervised video object segmentation (UVOS) aims at segmenting the primary objects in videos without any human intervention. Due to the lack of prior knowledge about the primary objects, identifying them from videos is the major challenge of UVOS. Previous methods often regard the moving objects as primary ones and rely on optical flow to capture the motion cues in videos, but the flow information alone is insufficient to distinguish the primary objects from the background objects that move together. This is because, when the noisy motion features are combined with the appearance features, the localization of the primary objects is …

Go to article

Cache-Efficient Fork-Processing Patterns On Large Graphs, Shengliang Lu, Shixuan Sun, Johns Paul, Yuchen Li, Bingsheng He Jun 2021

Cache-Efficient Fork-Processing Patterns On Large Graphs, Shengliang Lu, Shixuan Sun, Johns Paul, Yuchen Li, Bingsheng He

Research Collection School Of Computing and Information Systems

As large graph processing emerges, we observe a costly fork-processing pattern (FPP) that is common in many graph algorithms. The unique feature of the FPP is that it launches many independent queries from different source vertices on the same graph. For example, an algorithm in analyzing the network community profile can execute Personalized PageRanks that start from tens of thousands of source vertices at the same time. We study the efficiency of handling FPPs in state-of-the-art graph processing systems on multi-core architectures, including Ligra, Gemini, and GraphIt. We find that those systems suffer from severe cache miss penalty because of …

Go to article

Learning Contextual Causality Between Daily Events From Time-Consecutive Images, Hongming Zhang, Yintong Huo, Xinran Zhao, Yangqiu Song, Dan Roth Jun 2021

Learning Contextual Causality Between Daily Events From Time-Consecutive Images, Hongming Zhang, Yintong Huo, Xinran Zhao, Yangqiu Song, Dan Roth

Research Collection School Of Computing and Information Systems

Conventional textual-based causal knowledge acquisition methods typically require laborious and expensive human annotations. As a result, their scale is often limited. Moreover, as no context is provided during the annotation, the resulting causal knowledge records (e.g., ConceptNet) typically do not consider the context. In this paper, we move out of the textual domain to explore a more scalable way of acquiring causal knowledge and investigate the possibility of learning contextual causality from the visual signal. Specifically, we first propose a high-quality dataset Vis-Causal and then conduct experiments to demonstrate that with good language and visual representations, it is possible to …

Go to article

Projecting Your View Attentively: Monocular Road Scene Layout Estimation Via Cross-View Transformation, Weixiang Yang, Qi Li, Wenxi Liu, Yuanlong Yu, Yuexin Ma, Shengfeng He, Jia Pan Jun 2021

Projecting Your View Attentively: Monocular Road Scene Layout Estimation Via Cross-View Transformation, Weixiang Yang, Qi Li, Wenxi Liu, Yuanlong Yu, Yuexin Ma, Shengfeng He, Jia Pan

Research Collection School Of Computing and Information Systems

HD map reconstruction is crucial for autonomous driving. LiDAR-based methods are limited due to the deployed expensive sensors and time-consuming computation. Camera-based methods usually need to separately perform road segmentation and view transformation, which often causes distortion and the absence of content. To push the limits of the technology, we present a novel framework that enables reconstructing a local map formed by road layout and vehicle occupancy in the bird's-eye view given a front-view monocular image only. In particular, we propose a cross-view transformation module, which takes the constraint of cycle consistency between views into account and makes full use …

Go to article

Generating Face Images With Attributes For Free, Yaoyao Liu, Qianru Sun, He Xiangnan, Liu An-An, Su Yuting, Chua Tat-Seng Jun 2021

Generating Face Images With Attributes For Free, Yaoyao Liu, Qianru Sun, He Xiangnan, Liu An-An, Su Yuting, Chua Tat-Seng

Research Collection School Of Computing and Information Systems

With superhuman-level performance of face recognition, we are more concerned about the recognition of fine-grained attributes, such as emotion, age, and gender. However, given that the label space is extremely large and follows a long-tail distribution, it is quite expensive to collect sufficient samples for fine-grained attributes. This results in imbalanced training samples and inferior attribute recognition models. To this end, we propose the use of arbitrary attribute combinations, without human effort, to synthesize face images. In particular, to bridge the semantic gap between high-level attribute label space and low-level face image, we propose a novel neural-network-based approach that maps …

Go to article

Learning From The Master: Distilling Cross-Modal Advanced Knowledge For Lip Reading, Sucheng Ren, Yong Du, Jianming Lv, Guoqiang Han, Shengfeng He Jun 2021

Learning From The Master: Distilling Cross-Modal Advanced Knowledge For Lip Reading, Sucheng Ren, Yong Du, Jianming Lv, Guoqiang Han, Shengfeng He

Research Collection School Of Computing and Information Systems

Lip reading aims to predict the spoken sentences from silent lip videos. Due to the fact that such a vision task usually performs worse than its counterpart speech recognition, one potential scheme is to distill knowledge from a teacher pretrained by audio signals. However, the latent domain gap between the cross-modal data could lead to a learning ambiguity and thus limits the performance of lip reading. In this paper, we propose a novel collaborative framework for lip reading, and two aspects of issues are considered: 1) the teacher should understand bi-modal knowledge to possibly bridge the inherent cross-modal gap; 2) …

Go to article

First Prev Next Last