Open Access. Powered by Scholars. Published by Universities.®

Graphics and Human Computer Interfaces Commons™

Open Access. Powered by Scholars. Published by Universities.®

Singapore Management University

Discipline
Keyword
Publication Year
Publication
Publication Type

Articles 361 - 390 of 959

Full-Text Articles in Graphics and Human Computer Interfaces

Locally Varying Distance Transform For Unsupervised Visual Anomaly Detection, Wen-Yan Lin, Zhonghang Liu, Siying Liu Oct 2022

Locally Varying Distance Transform For Unsupervised Visual Anomaly Detection, Wen-Yan Lin, Zhonghang Liu, Siying Liu

Research Collection School Of Computing and Information Systems

Unsupervised anomaly detection on image data is notoriously unstable. We believe this is because many classical anomaly detectors implicitly assume data is low dimensional. However, image data is always high dimensional. Images can be projected to a low dimensional embedding but such projections rely on global transformations that truncate minor variations. As anomalies are rare, the final embedding often lacks the key variations needed to distinguish anomalies from normal instances. This paper proposes a new embedding using a set of locally varying data projections, with each projection responsible for persevering the variations that distinguish a local cluster of instances from …

Go to article

Interactive Contrastive Learning For Self-Supervised Entity Alignment, Kaisheng Zeng, Zhenhao Dong, Lei Hou, Yixin Cao, Minghao Hu, Jifan Yu, Xin Lv, Lei Cao, Xin Wang, Haozhuang Liu, Yi Huang, Jing Wan, Juanzi Li Oct 2022

Interactive Contrastive Learning For Self-Supervised Entity Alignment, Kaisheng Zeng, Zhenhao Dong, Lei Hou, Yixin Cao, Minghao Hu, Jifan Yu, Xin Lv, Lei Cao, Xin Wang, Haozhuang Liu, Yi Huang, Jing Wan, Juanzi Li

Research Collection School Of Computing and Information Systems

Self-supervised entity alignment (EA) aims to link equivalent entities across different knowledge graphs (KGs) without the use of pre-aligned entity pairs. The current state-of-the-art (SOTA) selfsupervised EA approach draws inspiration from contrastive learning, originally designed in computer vision based on instance discrimination and contrastive loss, and suffers from two shortcomings. Firstly, it puts unidirectional emphasis on pushing sampled negative entities far away rather than pulling positively aligned pairs close, as is done in the well-established supervised EA. Secondly, it advocates the minimum information requirement for self-supervised EA, while we argue that self-described KG’s side information (e.g., entity name, relation name, …

Go to article

Investigating Accessibility Challenges And Opportunities For Users With Low Vision Disabilities In Customer-To-Customer (C2c) Marketplaces, Bektur Ryskeldiev, Kotaro Hara, Mariko Kobayashi, Koki Kusano Oct 2022

Investigating Accessibility Challenges And Opportunities For Users With Low Vision Disabilities In Customer-To-Customer (C2c) Marketplaces, Bektur Ryskeldiev, Kotaro Hara, Mariko Kobayashi, Koki Kusano

Research Collection School Of Computing and Information Systems

Inaccessible e-commerce websites and mobile applications exclude people with visual impairments (PVI) from online shopping. Customer-to-customer (C2C) marketplaces, a form of e-commerce where trading happens not between businesses and customers but between customers, could pose a unique set of challenges in the interactions that the platform brings about. Through online questionnaire and remote interviews, we investigate problems experienced by people with low vision disabilities in common C2C scenarios. Our study with low vision participants (N = 12) reveal both previously known general accessibility issues (e.g., web and mobile interface accessibility) and C2C specific accessibility issues (e.g., inability to confirm item …

Go to article

Equivariance And Invariance Inductive Bias For Learning From Insufficient Data, Tan Wang, Qianru Sun, Sugiri Pranata, Karlekar Jayashree, Hanwang Zhang Oct 2022

Equivariance And Invariance Inductive Bias For Learning From Insufficient Data, Tan Wang, Qianru Sun, Sugiri Pranata, Karlekar Jayashree, Hanwang Zhang

Research Collection School Of Computing and Information Systems

We are interested in learning robust models from insufficient data, without the need for any externally pre-trained model checkpoints. First, compared to sufficient data, we show why insufficient data renders the model more easily biased to the limited training environments that are usually different from testing. For example, if all the training "swan" samples are "white", the model may wrongly use the "white" environment to represent the intrinsic class "swan". Then, we justify that equivariance inductive bias can retain the class feature while invariance inductive bias can remove the environmental feature, leaving only the class feature that generalizes to any …

Go to article

Pixel-Wise Energy-Biased Abstention Learning For Anomaly Segmentation On Complex Urban Driving Scenes, Yu Tian, Yuyuan Liu, Guansong Pang, Fengbei Liu, Yuanhong Chen, Gustavo Carneiro Oct 2022

Pixel-Wise Energy-Biased Abstention Learning For Anomaly Segmentation On Complex Urban Driving Scenes, Yu Tian, Yuyuan Liu, Guansong Pang, Fengbei Liu, Yuanhong Chen, Gustavo Carneiro

Research Collection School Of Computing and Information Systems

State-of-the-art (SOTA) anomaly segmentation approaches on complex urban driving scenes explore pixel-wise classification uncertainty learned from outlier exposure, or external reconstruction models. However, previous uncertainty approaches that directly associate high uncertainty to anomaly may sometimes lead to incorrect anomaly predictions, and external reconstruction models tend to be too inefficient for real-time self-driving embedded systems. In this paper, we propose a new anomaly segmentation method, named pixel-wise energy-biased abstention learning (PEBAL), that explores pixel-wise abstention learning (AL) with a model that learns an adaptive pixel-level anomaly class, and an energy-based model (EBM) that learns inlier pixel distribution. More specifically, PEBAL is …

Go to article

Learning Discriminative Representations Via Variational Self-Distillation For Cross-View Geo-Localization, Qian Hu, Wansi Li, Xing Xu, Ning Liu, Lei Wang Oct 2022

Learning Discriminative Representations Via Variational Self-Distillation For Cross-View Geo-Localization, Qian Hu, Wansi Li, Xing Xu, Ning Liu, Lei Wang

Research Collection School Of Computing and Information Systems

Cross-view geo-localization is to localize the same geographic target in images from different perspectives, e.g., satellite-view and drone-view. The primary challenge faced by existing methods is the large visual appearance changes across views. Most previous work utilizes the deep neural network to obtain the discriminative representations and directly uses them to accomplish the geo-localization task. However, these approaches ignore that the redundancy retained in the extracted features negatively impacts the result. In this paper, we argue that the information bottleneck (IB) can retain the most relevant information while removing as much redundancy as possible. The variational self-distillation (VSD) strategy provides …

Go to article

Unsupervised Video Hashing With Multi-Granularity Contextualization And Multi-Structure Preservation, Yanbin Hao, Jingru Duan, Hao Zhang, Bin Zhu, Pengyuan Zhou, Xiangnan He Oct 2022

Unsupervised Video Hashing With Multi-Granularity Contextualization And Multi-Structure Preservation, Yanbin Hao, Jingru Duan, Hao Zhang, Bin Zhu, Pengyuan Zhou, Xiangnan He

Research Collection School Of Computing and Information Systems

Unsupervised video hashing typically aims to learn a compact binary vector to represent complex video content without using manual annotations. Existing unsupervised hashing methods generally suffer from incomplete exploration of various perspective dependencies (e.g., long-range and short-range) and data structures that exist in visual contents, resulting in less discriminative hash codes. In this paper, we propose aMulti-granularity Contextualized and Multi-Structure preserved Hashing (MCMSH) method, exploring multiple axial contexts for discriminative video representation generation and various structural information for unsupervised learning simultaneously. Specifically, we delicately design three self-gating modules to separately model three granularities of dependencies (i.e., long/middle/short-range dependencies) and densely …

Go to article

Dualformer: Local-Global Stratified Transformer For Efficient Video Recognition, Yuxuan Liang, Pan Zhou, Roger Zimmermann, Shuicheng Yan Oct 2022

Dualformer: Local-Global Stratified Transformer For Efficient Video Recognition, Yuxuan Liang, Pan Zhou, Roger Zimmermann, Shuicheng Yan

Research Collection School Of Computing and Information Systems

While transformers have shown great potential on video recognition with their strong capability of capturing long-range dependencies, they often suffer high computational costs induced by the self-attention to the huge number of 3D tokens. In this paper, we present a new transformer architecture termed DualFormer, which can efficiently perform space-time attention for video recognition. Concretely, DualFormer stratifies the full space-time attention into dual cascaded levels, i.e., to first learn fine-grained local interactions among nearby 3D tokens, and then to capture coarse-grained global dependencies between the query token and global pyramid contexts. Different from existing methods that apply space-time factorization or …

Go to article

Self-Promoted Supervision For Few-Shot Transformer, Bowen Dong, Pan Zhou, Shuicheng Yan, Wangmeng Zuo Oct 2022

Self-Promoted Supervision For Few-Shot Transformer, Bowen Dong, Pan Zhou, Shuicheng Yan, Wangmeng Zuo

Research Collection School Of Computing and Information Systems

The few-shot learning ability of vision transformers (ViTs) is rarely investigated though heavily desired. In this work, we empirically find that with the same few-shot learning frameworks, e.g. MetaBaseline, replacing the widely used CNN feature extractor with a ViT model often severely impairs few-shot classification performance. Moreover, our empirical study shows that in the absence of inductive bias, ViTs often learn the low-qualified token dependencies under few-shot learning regime where only a few labeled training data are available, which largely contributes to the above performance degradation. To alleviate this issue, for the first time, we propose a simple yet effective …

Go to article

Video Graph Transformer For Video Question Answering, Junbin Xiao, Pan Zhou, Tat-Seng Chua, Shuicheng Yan Oct 2022

Video Graph Transformer For Video Question Answering, Junbin Xiao, Pan Zhou, Tat-Seng Chua, Shuicheng Yan

Research Collection School Of Computing and Information Systems

This paper proposes a Video Graph Transformer (VGT) model for Video Quetion Answering (VideoQA). VGT’s uniqueness are two-fold: 1) it designs a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations, and dynamics for complex spatio-temporal reasoning; and 2) it exploits disentangled video and text Transformers for relevance comparison between the video and text to perform QA, instead of entangled crossmodal Transformer for answer classification. Vision-text communication is done by additional cross-modal interaction modules. With more reasonable video encoding and QA solution, we show that VGT can achieve much better performances on VideoQA tasks …

Go to article

Mix-Dann And Dynamic-Modal-Distillation For Video Domain Adaptation, Yuehao Yin, Bin Zhu, Jingjing Chen, Lechao Cheng, Yu-Gang Jiang Oct 2022

Mix-Dann And Dynamic-Modal-Distillation For Video Domain Adaptation, Yuehao Yin, Bin Zhu, Jingjing Chen, Lechao Cheng, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Video domain adaptation is non-trivial due to video is inherently involved with multi-dimensional and multi-modal information. Existing works mainly adopt adversarial learning and self-supervised tasks to align features. Nevertheless, the explicit interaction between source and target in the temporal dimension, as well as the adaptation between modalities, are unexploited. In this paper, we propose Mix-Domain-Adversarial Neural Network and Dynamic-Modal-Distillation (MD-DMD), a novel multi-modal adversarial learning framework for unsupervised video domain adaptation. Our approach incorporates the temporal information between source and target domains, as well as the diversity of adaptability between modalities. On the one hand, for every single modality, we …

Go to article

Ergo: Event Relational Graph Transformer For Document-Level Event Causality Identification, Meiqi Chen, Yixin Cao, Kunquan Deng, Mukai Li, Kun Wang, Jing Shao, Yan Zhang Oct 2022

Ergo: Event Relational Graph Transformer For Document-Level Event Causality Identification, Meiqi Chen, Yixin Cao, Kunquan Deng, Mukai Li, Kun Wang, Jing Shao, Yan Zhang

Research Collection School Of Computing and Information Systems

Document-level Event Causality Identification (DECI) aims to identify event-event causal relations in a document. Existing works usually build an event graph for global reasoning across multiple sentences. However, the edges between events have to be carefully designed through heuristic rules or external tools. In this paper, we propose a novel Event Relational Graph TransfOrmer (ERGO) framework1 for DECI, to ease the graph construction and improve it over the noisy edge issue. Different from conventional event graphs, we define a pair of events as a node and build a complete event relational graph without any prior knowledge or tools. This naturally …

Go to article

Interactive Video Corpus Moment Retrieval Using Reinforcement Learning, Zhixin Ma, Chong-Wah Ngo Oct 2022

Interactive Video Corpus Moment Retrieval Using Reinforcement Learning, Zhixin Ma, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Known-item video search is effective with human-in-the-loop to interactively investigate the search result and refine the initial query. Nevertheless, when the first few pages of results are swamped with visually similar items, or the search target is hidden deep in the ranked list, finding the know-item target usually requires a long duration of browsing and result inspection. This paper tackles the problem by reinforcement learning, aiming to reach a search target within a few rounds of interaction by long-term learning from user feedbacks. Specifically, the system interactively plans for navigation path based on feedback and recommends a potential target that …

Go to article

On The Effectiveness Of Using Graphics Interrupt As A Side Channel For User Behavior Snooping, Haoyu Ma, Jianwen Tian, Debin Gao, Chunfu Jia Sep 2022

On The Effectiveness Of Using Graphics Interrupt As A Side Channel For User Behavior Snooping, Haoyu Ma, Jianwen Tian, Debin Gao, Chunfu Jia

Research Collection School Of Computing and Information Systems

Graphics Processing Units (GPUs) are now a key component of many devices and systems, including those in the cloud and data centers, thus are also subject to side-channel attacks. Existing side-channel attacks on GPUs typically leak information from graphics libraries like OpenGL and CUDA, which require creating contentions within the GPU resource space and are being mitigated with software patches. This paper evaluates potential side channels exposed at a lower-level interface between GPUs and CPUs, namely the graphics interrupts. These signals could indicate unique signatures of GPU workload, allowing a spy process to infer the behavior of other processes. We …

Go to article

Hierarchical Semantic-Aware Neural Code Representation, Yuan Jiang, Xiaohong Su, Christoph Treude, Tiantian Wang Sep 2022

Hierarchical Semantic-Aware Neural Code Representation, Yuan Jiang, Xiaohong Su, Christoph Treude, Tiantian Wang

Research Collection School Of Computing and Information Systems

Code representation is a fundamental problem in many software engineering tasks. Despite the effort made by many researchers, it is still hard for existing methods to fully extract syntactic, structural and sequential features of source code, which form the hierarchical semantics of the program and are necessary to achieve a deeper code understanding. To alleviate this difficulty, we propose a new supervised approach based on the novel use of Tree-LSTM to incorporate the sequential and the global semantic features of programs explicitly into the representation model. Unlike previous techniques, our proposed model can not only learn low-level syntactic information within …

Go to article

Joint Hyperbolic And Euclidean Geometry Contrastive Graph Neural Networks, Xiaoyu Xu, Guansong Pang, Di Wu, Mingsheng Shang Sep 2022

Joint Hyperbolic And Euclidean Geometry Contrastive Graph Neural Networks, Xiaoyu Xu, Guansong Pang, Di Wu, Mingsheng Shang

Research Collection School Of Computing and Information Systems

Graph Neural Networks (GNNs) have demonstrated state-of-the-art performance in a wide variety of analytical tasks. Current GNN approaches focus on learning representations in a Euclidean space, which are effective in capturing non-tree-like structural relations, but they fail to model complex relations in many real-world graphs, such as tree-like hierarchical graph structure. This paper instead proposes to learn representations in both Euclidean and hyperbolic spaces to model these two types of graph geometries. To this end, we introduce a novel approach - Joint hyperbolic and Euclidean geometry contrastive graph neural networks (JointGMC). JointGMC is enforced to learn multiple layer-wise optimal combinations …

Go to article

Largeea: Aligning Entities For Large-Scale Knowledge Graphs, Congcong Ge, Xiaoze Liu, Lu Chen, Yunjun Gao, Baihua Zheng Sep 2022

Largeea: Aligning Entities For Large-Scale Knowledge Graphs, Congcong Ge, Xiaoze Liu, Lu Chen, Yunjun Gao, Baihua Zheng

Research Collection School Of Computing and Information Systems

Entity alignment (EA) aims to find equivalent entities in different knowledge graphs (KGs). Current EA approaches suffer from scalability issues, limiting their usage in real-world EA scenarios. To tackle this challenge, we propose LargeEA to align entities between large-scale KGs. LargeEA consists of two channels, i.e., structure channel and name channel. For the structure channel, we present METIS-CPS, a memory-saving mini-batch generation strategy, to partition large KGs into smaller mini-batches. LargeEA, designed as a general tool, can adopt any existing EA approach to learn entities’ structural features within each mini-batch independently. For the name channel, we first introduce NFF, a …

Go to article

Eeg-Video Emotion-Based Summarization: Learning With Eeg Auxiliary Signals, Wai-Cheong L. Lew, Di Wang, Kai Keng Ang, Joo-Hwee Lim, Chai Quek, Ah-Hwee Tan Sep 2022

Eeg-Video Emotion-Based Summarization: Learning With Eeg Auxiliary Signals, Wai-Cheong L. Lew, Di Wang, Kai Keng Ang, Joo-Hwee Lim, Chai Quek, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

Video summarization is the process of selecting a subset of informative keyframes to expedite storytelling with limited loss of information. In this paper, we propose an EEG-Video Emotion-based Summarization (EVES) model based on a multimodal deep reinforcement learning (DRL) architecture that leverages neural signals to learn visual interestingness to produce quantitatively and qualitatively better video summaries. As such, EVES does not learn from the expensive human annotations but the multimodal signals. Furthermore, to ensure the temporal alignment and minimize the modality gap between the visual and EEG modalities, we introduce a Time Synchronization Module (TSM) that uses an attention mechanism …

Go to article

Developing Gamification Research In Information Systems, De Liu, Richard Landers, Paul Lowry, Fiona Fui-Hoon Nah, Radhika Santhanam Aug 2022

Developing Gamification Research In Information Systems, De Liu, Richard Landers, Paul Lowry, Fiona Fui-Hoon Nah, Radhika Santhanam

Research Collection School Of Computing and Information Systems

Gamification, an umbrella term for incorporating game design elements into non-game systems to make them more engaging and productive, is of interest to Information System (IS) communities, because it has wide applications, such as, gamification designs for workplace, learning and health apps, social media, online communities, and the gig economy. As a multidisciplinary research domain, gamification integrates elements of technology, human motivation, task design, human-computer-interface design, and algorithms/AI design, and is a fertile ground for IS researchers with a variety of different background. There are many challenges in conducting gamification research, including how to get started in this highly multidisciplinary …

Go to article

Enhancing Security Patch Identification By Capturing Structures In Commits, Bozhi Wu, Shangqing Liu, Ruitao Feng, Xiaofei Xie, Jingkai Siow, Shang-Wei Lin Jul 2022

Enhancing Security Patch Identification By Capturing Structures In Commits, Bozhi Wu, Shangqing Liu, Ruitao Feng, Xiaofei Xie, Jingkai Siow, Shang-Wei Lin

Research Collection School Of Computing and Information Systems

With the rapid increasing number of open source software (OSS), the majority of the software vulnerabilities in the open source components are fixed silently, which leads to the deployed software that integrated them being unable to get a timely update. Hence, it is critical to design a security patch identification system to ensure the security of the utilized software. However, most of the existing works for security patch identification just consider the changed code and the commit message of a commit as a flat sequence of tokens with simple neural networks to learn its semantics, while the structure information is …

Go to article

Prototypical Graph Contrastive Learning, Shuai Lin, Chen Liu, Pan Zhou, Zi-Yuan Hu, Shuojia Wang, Ruihui Zhao, Yefeng Zheng, Liang Lin, Eric Xing, Xiaodan Liang Jul 2022

Prototypical Graph Contrastive Learning, Shuai Lin, Chen Liu, Pan Zhou, Zi-Yuan Hu, Shuojia Wang, Ruihui Zhao, Yefeng Zheng, Liang Lin, Eric Xing, Xiaodan Liang

Research Collection School Of Computing and Information Systems

Graph-level representations are critical in various real-world applications, such as predicting the properties of molecules. But in practice, precise graph annotations are generally very expensive and time-consuming. To address this issue, graph contrastive learning constructs instance discrimination task which pulls together positive pairs (augmentation pairs of the same graph) and pushes away negative pairs (augmentation pairs of different graphs) for unsupervised representation learning. However, since for a query, its negatives are uniformly sampled from all graphs, existing methods suffer from the critical sampling bias issue, i.e., the negatives likely having the same semantic structure with the query, leading to performance …

Go to article

End-To-End Open-Set Semi-Supervised Node Classification With Out-Of-Distribution Detection, Tiancheng Huang, Donglin Wang, Yuan Fang Jul 2022

End-To-End Open-Set Semi-Supervised Node Classification With Out-Of-Distribution Detection, Tiancheng Huang, Donglin Wang, Yuan Fang

Research Collection School Of Computing and Information Systems

Out-Of-Distribution (OOD) samples are prevalent in real-world applications. The OOD issue becomes even more severe on graph data, as the effect of OOD nodes can be potentially amplified by propagation through the graph topology. Recent works have considered the OOD detection problem, which is critical for reducing the uncertainty in learning and improving the robustness. However, no prior work considers simultaneously OOD detection and node classification on graphs in an end-to-end manner. In this paper, we study a novel problem of end-to-end open-set semisupervised node classification (OSSNC) on graphs, which deals with node classification in the presence of OOD nodes. …

Go to article

Proceedings Of The 13th International Workshop On Graph Computation Models (Gcm 2022), Reiko Heckel, Christopher M. Poskitt Jul 2022

Proceedings Of The 13th International Workshop On Graph Computation Models (Gcm 2022), Reiko Heckel, Christopher M. Poskitt

Research Collection School Of Computing and Information Systems

This volume contains the proceedings of the Thirteenth International Workshop on Graph Computation Models (GCM 2022) , which was held in Nantes, France on 6th July 2022 as part of the STAF federation of conferences. Graphs are common mathematical structures that are visual and intuitive. They constitute a natural and seamless way for system modelling in science, engineering, and beyond, including computer science, biology, and business process modelling. Graph computation models constitute a class of very high-level models where graphs are first-class citizens. The aim of the International GCM Workshop series is to bring together researchers interested in all aspects …

Go to article

A Low-Cost Virtual Coach For 2d Video-Based Compensation Assessment Of Upper Extremity Rehabilitation Exercises, Ana Rita Coias, Min Hun Lee, Alexandre Bernardino Jul 2022

A Low-Cost Virtual Coach For 2d Video-Based Compensation Assessment Of Upper Extremity Rehabilitation Exercises, Ana Rita Coias, Min Hun Lee, Alexandre Bernardino

Research Collection School Of Computing and Information Systems

Background: The increasing demands concerning stroke rehabilitation and in-home exercise promotion grew the need for affordable and accessible assistive systems to promote patients' compliance in therapy. These assistive systems require quantitative methods to assess patients' quality of movement and provide feedback on their performance. However, state-of-the-art quantitative assessment approaches require expensive motion-capture devices, which might be a barrier to the development of low-cost systems.Methods: In this work, we develop a low-cost virtual coach (VC) that requires only a laptop with a webcam to monitor three upper extremity rehabilitation exercises and provide real-time visual and audio feedback on compensatory motion patterns …

Go to article

Towards Improving System Performance In Large Scale Multi-Agent Systems With Selfish Agents, Rajiv Ranjan Kumar Jul 2022

Towards Improving System Performance In Large Scale Multi-Agent Systems With Selfish Agents, Rajiv Ranjan Kumar

Dissertations and Theses Collection (Open Access)

Intelligent agents are becoming increasingly prevalent in a wide variety of domains including but not limited to transportation, safety and security. To better utilize the intelligence, there has been increasing focus on frameworks and methods for coordinating these intelligent agents. This thesis is specifically targeted at providing solution approaches for improving large scale multi-agent systems with selfish intelligent agents. In such systems, the performance of an agent depends on not just his/her own efforts, but also on other agent’s decisions. The complexity of interactions among multiple agents, coupled with the large scale nature of the problem domains and the uncertainties …

Go to article

Docee: A Large-Scale And Fine-Grained Benchmark For Document-Level Event Extraction, Meihan Tong, Bin Xu, Shuai Wang, Meihuan Han, Yixin Cao, Jiangqi Zhu, Siyu Chen, Lei Hou, Juanzi Li Jul 2022

Docee: A Large-Scale And Fine-Grained Benchmark For Document-Level Event Extraction, Meihan Tong, Bin Xu, Shuai Wang, Meihuan Han, Yixin Cao, Jiangqi Zhu, Siyu Chen, Lei Hou, Juanzi Li

Research Collection School Of Computing and Information Systems

Event extraction aims to identify an event and then extract the arguments participating in the event. Despite the great success in sentencelevel event extraction, events are more naturally presented in the form of documents, with event arguments scattered in multiple sentences. However, a major barrier to promote documentlevel event extraction has been the lack of large-scale and practical training and evaluation datasets. In this paper, we present DocEE, a new document-level event extraction dataset including 27,000+ events, 180,000+ arguments. We highlight three features: largescale manual annotations, fine-grained argument types and application-oriented settings. Experiments show that there is still a big …

Go to article

Multi-View Scheduling Of Onboard Live Video Analytics To Minimize Frame Processing Latency, Shengzhong Liu, Tianshi Wang, Hongpeng Guo, Xinzhe Fu, Philip David, Maggie Wigness, Archan Misra, Tarek Abdelzaher Jun 2022

Multi-View Scheduling Of Onboard Live Video Analytics To Minimize Frame Processing Latency, Shengzhong Liu, Tianshi Wang, Hongpeng Guo, Xinzhe Fu, Philip David, Maggie Wigness, Archan Misra, Tarek Abdelzaher

Research Collection School Of Computing and Information Systems

This paper presents a real-time multi-view scheduling framework for DNN-based live video analytics at the edge to minimize frame processing latency. The work is motivated by applications where a higher frame rate is important, not to miss actions of interest. Examples include defense, border security, and intruder detection applications where sensors (in this paper, cameras) are deployed to monitor key roads, chokepoints, or passageways to identify events of interest (and intervene in real-time). Supporting a higher frame rate entails lowering frame processing latency. We assume that multiple cameras are deployed with partially overlapping views. Each camera has access to limited …

Go to article

Class Re-Activation Maps For Weakly-Supervised Semantic Segmentation, Zhaozheng Chen, Tan Wang, Xiongwei Wu, Xian-Sheng Hua, Hanwang Zhang, Qianru Sun Jun 2022

Class Re-Activation Maps For Weakly-Supervised Semantic Segmentation, Zhaozheng Chen, Tan Wang, Xiongwei Wu, Xian-Sheng Hua, Hanwang Zhang, Qianru Sun

Research Collection School Of Computing and Information Systems

Extracting class activation maps (CAM) is arguably the most standard step of generating pseudo masks for weakly supervised semantic segmentation (WSSS). Yet, we find that the crux of the unsatisfactory pseudo masks is the binary cross-entropy loss (BCE) widely used in CAM. Specifically, due to the sum-over-class pooling nature of BCE, each pixel in CAM may be responsive to multiple classes co-occurring in the same receptive field. To this end, we introduce an embarrassingly simple yet surprisingly effective method: Reactivating the converged CAM with BCE by using softmax crossentropy loss (SCE), dubbed ReCAM. Given an image, we use CAM to …

Go to article

Metaformer Is Actually What You Need For Vision, Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xinchao Wang, Jiashi Feng, Shuicheng Yan Jun 2022

Metaformer Is Actually What You Need For Vision, Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xinchao Wang, Jiashi Feng, Shuicheng Yan

Research Collection School Of Computing and Information Systems

Transformers have shown great potential in computer vision tasks. A common belief is their attention-based token mixer module contributes most to their competence. However, recent works show the attention-based module in transformers can be replaced by spatial MLPs and the resulted models still perform quite well. Based on this observation, we hypothesize that the general architecture of the transformers, instead of the specific token mixer module, is more essential to the model's performance. To verify this, we deliberately replace the attention module in transformers with an embarrassingly simple spatial pooling operator to conduct only basic token mixing. Surprisingly, we observe …

Go to article

Faithful Extreme Rescaling Via Generative Prior Reciprocated Invertible Representations, Zhixuan Zhong, Liangyu Chai, Yang Zhou, Bailin Deng, Jia Pan, Shengfeng He Jun 2022

Faithful Extreme Rescaling Via Generative Prior Reciprocated Invertible Representations, Zhixuan Zhong, Liangyu Chai, Yang Zhou, Bailin Deng, Jia Pan, Shengfeng He

Research Collection School Of Computing and Information Systems

This paper presents a Generative prior ReciprocAted Invertible rescaling Network (GRAIN) for generating faithful high-resolution (HR) images from low-resolution (LR) invertible images with an extreme upscaling factor (64×). Previous researches have leveraged the prior knowledge of a pretrained GAN model to generate high-quality upscaling results. However, they fail to produce pixel-accurate results due to the highly ambiguous extreme mapping process. We remedy this problem by introducing a reciprocated invertible image rescaling process, in which high-resolution information can be delicately embedded into an invertible low-resolution image and generative prior for a faithful HR reconstruction. In particular, the invertible LR features not …

Go to article

First Prev Next Last