Open Access. Powered by Scholars. Published by Universities.®

Graphics and Human Computer Interfaces Commons™

Open Access. Powered by Scholars. Published by Universities.®

Singapore Management University

Discipline
Keyword
Publication Year
Publication
Publication Type

Articles 391 - 420 of 959

Full-Text Articles in Graphics and Human Computer Interfaces

Group Contextualization For Video Recognition, Yanbin Hao, Hao Zhang, Chong-Wah Ngo, Xiangnan He Jun 2022

Group Contextualization For Video Recognition, Yanbin Hao, Hao Zhang, Chong-Wah Ngo, Xiangnan He

Research Collection School Of Computing and Information Systems

Learning discriminative representation from the complex spatio-temporal dynamic space is essential for video recognition. On top of those stylized spatio-temporal computational units, further refining the learnt feature with axial contexts is demonstrated to be promising in achieving this goal. However, previous works generally focus on utilizing a single kind of contexts to calibrate entire feature channels and could hardly apply to deal with diverse video activities. The problem can be tackled by using pair-wise spatio-temporal attentions to recompute feature response with cross-axis contexts at the expense of heavy computations. In this paper, we propose an efficient feature refinement method that …

Go to article

Revisiting Local Descriptor For Improved Few-Shot Classification, Jun He, Richang Hong, Xueliang Liu, Mingliang Xu, Qianru Sun Jun 2022

Revisiting Local Descriptor For Improved Few-Shot Classification, Jun He, Richang Hong, Xueliang Liu, Mingliang Xu, Qianru Sun

Research Collection School Of Computing and Information Systems

Few-shot classification studies the problem of quickly adapting a deep learner to understanding novel classes based on few support images. In this context, recent research efforts have been aimed at designing more and more complex classifiers that measure similarities between query and support images but left the importance of feature embeddings seldom explored. We show that the reliance on sophisticated classifiers is not necessary, and a simple classifier applied directly to improved feature embeddings can instead outperform most of the leading methods in the literature. To this end, we present a new method, named DCAP, for few-shot classification, in which …

Go to article

Flavor-Videos: Enhancing The Flavor Perception Of Food While Eating With Videos, Meetha Nesam James, Nimesha Ranasinghe, Anthony Tang, Lora Oehlberg Jun 2022

Flavor-Videos: Enhancing The Flavor Perception Of Food While Eating With Videos, Meetha Nesam James, Nimesha Ranasinghe, Anthony Tang, Lora Oehlberg

Research Collection School Of Computing and Information Systems

People are typically involved in different activities while eating, particularly when eating alone, such as watching television or playing games on their phones. Previous research in Human-Food Interaction (HFI) has primarily focused on studying people’s motivation and analyzing of the media content watched while eating. However, their impact on human behavioral and cognitive processes, particularly flavor perception and its attributes, remains underexplored. We present a user study to investigate the influence of six types of videos, including mukbang – a new food video genre, on flavor perceptions (taste sensations, liking, and emotions) while eating plain white rice. Our findings revealed …

Go to article

Mlp-3d: A Mlp-Like 3d Architecture With Grouped Time Mixing, Zhaofan Qiu, Ting Yao, Chong-Wah Ngo, Tao Mei Jun 2022

Mlp-3d: A Mlp-Like 3d Architecture With Grouped Time Mixing, Zhaofan Qiu, Ting Yao, Chong-Wah Ngo, Tao Mei

Research Collection School Of Computing and Information Systems

Convolutional Neural Networks (CNNs) have been re-garded as the go-to models for visual recognition. More re-cently, convolution-free networks, based on multi-head self-attention (MSA) or multi-layer perceptrons (MLPs), become more and more popular. Nevertheless, it is not trivial when utilizing these newly-minted networks for video recognition due to the large variations and complexities in video data. In this paper, we present MLP-3D networks, a novel MLP-like 3D architecture for video recognition. Specifically, the architecture consists of MLP-3D blocks, where each block contains one MLP applied across tokens (i.e., token-mixing MLP) and one MLP applied independently to each token (i.e., channel MLP). …

Go to article

Adaptive Temporal Grouping For Black-Box Adversarial Attacks On Videos, Zhipeng Wei, Jingjing Chen, Hao Zhang, Linxi Jiang, Yu-Gang Jiang Jun 2022

Adaptive Temporal Grouping For Black-Box Adversarial Attacks On Videos, Zhipeng Wei, Jingjing Chen, Hao Zhang, Linxi Jiang, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Deep-learning based video models, which have remarkable performance on action recognition tasks, are recently proved to be vulnerable to adversarial samples, even those generated in the black-box setting. However, these black-box attack methods are insufficient to attack videos models in real-world applications due to the requirement of lots of queries. To this end, we propose to boost the efficiency of black-box attacks on video recognition models. Although videos carry rich temporal information, they include redundant spatial information from adjacent frames. This motivates us to introduce the adaptive temporal grouping (ATG) method, which groups video frames by the similarity of their …

Go to article

Multimodal Zero-Shot Hateful Meme Detection, Jiawen Zhu, Roy Ka-Wei Lee, Wen Haw Chong Jun 2022

Multimodal Zero-Shot Hateful Meme Detection, Jiawen Zhu, Roy Ka-Wei Lee, Wen Haw Chong

Research Collection School Of Computing and Information Systems

Facebook has recently launched the hateful meme detection challenge, which garnered much attention in academic and industry research communities. Researchers have proposed multimodal deep learning classification methods to perform hateful meme detection. While the proposed methods have yielded promising results, these classification methods are mostly supervised and heavily rely on labeled data that are not always available in the real-world setting. Therefore, this paper explores and aims to perform hateful meme detection in a zero-shot setting. Working towards this goal, we propose Target-Aware Multimodal Enhancement (TAME), which is a novel deep generative framework that can improve existing hateful meme classification …

Go to article

A Simple Data Mixing Prior For Improving Self-Supervised Learning, Sucheng Ren, Huiyu Wang, Zhengqi Gao, Shengfeng He, Alan Yuille, Yuyin Zhou, Cihang Xie Jun 2022

A Simple Data Mixing Prior For Improving Self-Supervised Learning, Sucheng Ren, Huiyu Wang, Zhengqi Gao, Shengfeng He, Alan Yuille, Yuyin Zhou, Cihang Xie

Research Collection School Of Computing and Information Systems

Data mixing (e.g., Mixup, Cutmix, ResizeMix) is an essential component for advancing recognition models. In this paper, we focus on studying its effectiveness in the self-supervised setting. By noticing the mixed images that share the same source images are intrinsically related to each other, we hereby propose SDMP, short for Simple Data Mixing Prior, to capture this straightforward yet essential prior, and position such mixed images as additional positive pairs to facilitate self-supervised representation learning. Our experiments verify that the proposed SDMP enables data mixing to help a set of self-supervised learning frameworks (e.g., MoCo) achieve better accuracy and out-of-distribution …

Go to article

High-Resolution Face Swapping Via Latent Semantics Disentanglement, Yangyang Xu, Bailin Deng, Junle Wang, Yanqing Jing, Jia Pan, Shengfeng He Jun 2022

High-Resolution Face Swapping Via Latent Semantics Disentanglement, Yangyang Xu, Bailin Deng, Junle Wang, Yanqing Jing, Jia Pan, Shengfeng He

Research Collection School Of Computing and Information Systems

We present a novel high-resolution face swapping method using the inherent prior knowledge of a pre-trained GAN model. Although previous research can leverage generative priors to produce high-resolution results, their quality can suffer from the entangled semantics of the latent space. We explicitly disentangle the latent semantics by utilizing the progressive nature of the generator, deriving structure at-tributes from the shallow layers and appearance attributes from the deeper ones. Identity and pose information within the structure attributes are further separated by introducing a landmark-driven structure transfer latent direction. The disentangled latent code produces rich generative features that incorporate feature blending …

Go to article

Cross-Lingual Adaptation For Recipe Retrieval With Mixup, Bin Zhu, Chong-Wah Ngo, Jingjing Chen, Wing-Kwong Chan Jun 2022

Cross-Lingual Adaptation For Recipe Retrieval With Mixup, Bin Zhu, Chong-Wah Ngo, Jingjing Chen, Wing-Kwong Chan

Research Collection School Of Computing and Information Systems

Cross-modal recipe retrieval has attracted research attention in recent years, thanks to the availability of large-scale paired data for training. Nevertheless, obtaining adequate recipe-image pairs covering the majority of cuisines for supervised learning is difficult if not impossible. By transferring knowledge learnt from a data-rich cuisine to a data-scarce cuisine, domain adaptation sheds light on this practical problem. Nevertheless, existing works assume recipes in source and target domains are mostly originated from the same cuisine and written in the same language. This paper studies unsupervised domain adaptation for image-to-recipe retrieval, where recipes in source and target domains are in different …

Go to article

Reinforcement Learning-Based Interactive Video Search, Zhixin Ma, Jiaxin Wu, Zhijian Hou, Chong-Wah Ngo Jun 2022

Reinforcement Learning-Based Interactive Video Search, Zhixin Ma, Jiaxin Wu, Zhijian Hou, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Despite the rapid progress in text-to-video search due to the advancement of cross-modal representation learning, the existing techniques still fall short in helping users to rapidly identify the search targets. Particularly, in the situation that a system suggests a long list of similar candidates, the user needs to painstakingly inspect every search result. The experience is frustrated with repeated watching of similar clips, and more frustratingly, the search targets may be overlooked due to mental tiredness. This paper explores reinforcement learning-based (RL) searching to relieve the user from the burden of brute force inspection. Specifically, the system maintains a graph …

Go to article

Hci In Southeast Asia: The Journey Forward, E. Sari, J.A. Tedjasaputra, Y. Kurniawan, E. Zulaikha, A. Asfarian, M. Ghazali, A. Sivaji, J.A. Abu Bakar, C.Y. Wong, N.M. Norowi, Tamas Makany, D. Perera-Schulz, T. Chintakovid, S. Nuchitprasitchai, Ethel Ong May 2022

Hci In Southeast Asia: The Journey Forward, E. Sari, J.A. Tedjasaputra, Y. Kurniawan, E. Zulaikha, A. Asfarian, M. Ghazali, A. Sivaji, J.A. Abu Bakar, C.Y. Wong, N.M. Norowi, Tamas Makany, D. Perera-Schulz, T. Chintakovid, S. Nuchitprasitchai, Ethel Ong

Research Collection Lee Kong Chian School Of Business

SEACHI 2022 has been conducted to bring HCI and UX leaders in Southeast Asia to discuss the current state-of-the-art HCI and UX teaching, practice, and support they experience in their region. This activity aims to explore the potentials and challenges and identify the gaps amongst different sectors in different countries. Through this workshop, we will have a common understanding of what we face. It explores how we can work collaboratively to achieve a better purpose, i.e., to grow HCI and UX fields in Southeast Asia. This one-day online workshop was conducted as a collocated event of CHI 2022 and was …

Go to article

Designing Visuo-Haptic Illusions With Proxies In Virtual Reality: Exploration Of Grasp, Movement Trajectory And Object Mass, Martin Feick, Kora Persephone Regitz, Anthony Tang, Antonio Kruger May 2022

Designing Visuo-Haptic Illusions With Proxies In Virtual Reality: Exploration Of Grasp, Movement Trajectory And Object Mass, Martin Feick, Kora Persephone Regitz, Anthony Tang, Antonio Kruger

Research Collection School Of Computing and Information Systems

Visuo-haptic illusions are a method to expand proxy-based interactions in VR by introducing unnoticeable discrepancies between the virtual and real world. Yet how different design variables affect the illusions with proxies is still unclear. To unpack a subset of variables, we conducted two user studies with 48 participants to explore the impact of (1) different grasping types and movement trajectories, and (2) different grasping types and object masses on the discrepancy which may be introduced. Our Bayes analysis suggests that grasping types and object masses (≤ 500 g) did not noticeably affect the discrepancy, but for movement trajectory, results were …

Go to article

Tourgether360: Exploring 360° Tour Videos With Others, Kartikaeya Kumar, Lev Poretski, Jiannan Li, Anthony Tang May 2022

Tourgether360: Exploring 360° Tour Videos With Others, Kartikaeya Kumar, Lev Poretski, Jiannan Li, Anthony Tang

Research Collection School Of Computing and Information Systems

Contemporary 360° video players do not provide ways to let people explore the videos together. Tourgether360 addresses this problem for 360° tour videos using a pseudo-spatial navigation technique that provides both an overhead “context” view of the environment as a minimap, as well as a shared pseudo-3D environment for exploring the video. Collaborators appear as avatars along a track depending on their position in the video timeline and can point and synchronize their playback. In this work, we describe the intellectual precedents for this work, our design goals, and our implementation approach of Tourgether360. Finally, we discuss future work based …

Go to article

Mmekg: Multi-Modal Event Knowledge Graph Towards Universal Representation Across Modalities, Yubo Ma, Zehao Wang, Mukai Li, Yixin Cao, Meiqi Chen, Xinze Li, Wenqi Sun, Kunquan Deng, Kun Wang, Aixin Sun, Jing Shao May 2022

Mmekg: Multi-Modal Event Knowledge Graph Towards Universal Representation Across Modalities, Yubo Ma, Zehao Wang, Mukai Li, Yixin Cao, Meiqi Chen, Xinze Li, Wenqi Sun, Kunquan Deng, Kun Wang, Aixin Sun, Jing Shao

Research Collection School Of Computing and Information Systems

Events are fundamental building blocks of realworld happenings. In this paper, we present a large-scale, multi-modal event knowledge graph named MMEKG. MMEKG unifies different modalities of knowledge via events, which complement and disambiguate each other. Specifically, MMEKG incorporates (i) over 990 thousand concept events with 644 relation types to cover most types of happenings, and (ii) over 863 million instance events connected through 934 million relations, which provide rich contextual information in texts and/or images. To collect billion-scale instance events and relations among them, we additionally develop an efficient yet effective pipeline for textual/visual knowledge extraction system. We also develop …

Go to article

Prompt For Extraction? Paie: Prompting Argument Interaction For Event Argument Extraction, Yubo Ma, Zehao Wang, Yixin Cao, Mukai Li, Meiqi Chen, Kun Wang, Jing Shao May 2022

Prompt For Extraction? Paie: Prompting Argument Interaction For Event Argument Extraction, Yubo Ma, Zehao Wang, Yixin Cao, Mukai Li, Meiqi Chen, Kun Wang, Jing Shao

Research Collection School Of Computing and Information Systems

In this paper, we propose an effective yet efficient model PAIE for both sentence-level and document-level Event Argument Extraction (EAE), which also generalizes well when there is a lack of training data. On the one hand, PAIE utilizes prompt tuning for extractive objectives to take the best advantages of Pre-trained Language Models (PLMs). It introduces two span selectors based on the prompt to select start/end tokens among input texts for each role. On the other hand, it captures argument interactions via multi-role prompts and conducts joint optimization with optimal span assignments via a bipartite matching loss. Also, with a flexible …

Go to article

Topic-Guided Conversational Recommender In Multiple Domains, Lizi Liao, Ryuichi Takanobu, Yunshan Ma, Xun Yang, Minlie Huang, Tat-Seng Chua May 2022

Topic-Guided Conversational Recommender In Multiple Domains, Lizi Liao, Ryuichi Takanobu, Yunshan Ma, Xun Yang, Minlie Huang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Conversational systems have recently attracted significant attention. Both the research community and industry believe that it will exert huge impact on human-computer interaction, and specifically, the IR/RecSys community has begun to explore Conversational Recommendation. In real-life scenarios, such systems are often urgently needed in helping users accomplishing different tasks under various situations. However, existing works still face several shortcomings: (1) Most efforts are largely confined in single task setting. They fall short of hands in handling tasks across domains. (2) Aside from soliciting user preference from dialogue history, a conversational recommender naturally has access to the back-end data structure which …

Go to article

Hci Education And Ux Practice: Highlights From Singapore, Tamas Makany, Dharani Perera-Schulz May 2022

Hci Education And Ux Practice: Highlights From Singapore, Tamas Makany, Dharani Perera-Schulz

Research Collection Lee Kong Chian School Of Business

This position paper highlights trends in education, practice, and support of HCI/UX in Singapore, a small city-state island in Southeast Asia. The paper was prepared for the 2022 Southeast Asia Computer-Human Interaction (SEACHI'22) virtual workshop on Apr 14, 2022, as part of the ACM CHI Conference on Human Factors in Computing Systems (CHI'22) international conference.

Go to article

Cost-Effective And Collaborative Methods To Author Video's Scene Description For Blind People, Rosiana Natalie May 2022

Cost-Effective And Collaborative Methods To Author Video's Scene Description For Blind People, Rosiana Natalie

Research Collection School Of Computing and Information Systems

The majority of online video content remains inaccessible for blind people due to the lack of audio descriptions. Content creators have traditionally relied on professionals to author audio descriptions, but their service is costly and not readily available. In this research, I introduce four threads of research that I will conduct for my Ph.D. dissertation, aimed to create methods and tools that are both time- and cost-effective in providing good quality audio descriptions. They are: (i) The development and evaluation of mixed-ability collaboration authoring tool, (ii) The formative study to uncover the feedback pattern from the reviewer, (iii) the evaluation …

Go to article

Watch Your Flavors: Augmenting People's Flavor Perceptions And Associated Emotions Based On Videos Watched While Eating, Meetha Nesam James, Nimesha Ranasinghe, Anthony Tang, Lora Oehlberg May 2022

Watch Your Flavors: Augmenting People's Flavor Perceptions And Associated Emotions Based On Videos Watched While Eating, Meetha Nesam James, Nimesha Ranasinghe, Anthony Tang, Lora Oehlberg

Research Collection School Of Computing and Information Systems

People engage in different activities while eating alone, such as watching television or scrolling through social media on their phones. However, the impacts of these visual contents on human cognitive processes, particularly related to flavor perception and its attributes, are still not thoroughly explored. This paper presents a user study to evaluate the influence of six different types of video content (including nature, cooking, and a new food video genre known as mukbang) on people’s flavor perceptions in terms of taste sensations, liking, and emotions while eating plain white rice. Our findings revealed that the participants’ flavor perceptions are augmented …

Go to article

Understanding Crowdsourcing Requesters’ Wage Setting Behaviors, Kotaro Hara, Yudai Tanaka May 2022

Understanding Crowdsourcing Requesters’ Wage Setting Behaviors, Kotaro Hara, Yudai Tanaka

Research Collection School Of Computing and Information Systems

Requesters on crowdsourcing platforms like Amazon Mechanical Turk (AMT) compensate workers inadequately. One potential reason for the underpayment is that the AMT’s requester interface provides limited information about estimated wages, preventing requesters from knowing if they are offering a fair piece-rate reward. To assess if presenting wage information affects requesters’ reward setting behaviors, we conducted a controlled study with 63 participants. We had three levels for a between-subjects factor in a mixed design study, where we provided participants with: no wage information, wage point estimate, and wage distribution. Each participant had three stages of adjusting the reward and controlling the …

Go to article

Do Pre-Trained Models Benefit Knowledge Graph Completion? A Reliable Evaluation And A Reasonable Approach, Xin Lv, Yankai Lin, Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie Zhou May 2022

Do Pre-Trained Models Benefit Knowledge Graph Completion? A Reliable Evaluation And A Reasonable Approach, Xin Lv, Yankai Lin, Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie Zhou

Research Collection School Of Computing and Information Systems

In recent years, pre-trained language models (PLMs) have been shown to capture factual knowledge from massive texts, which encourages the proposal of PLM-based knowledge graph completion (KGC) models. However, these models are still quite behind the SOTA KGC models in terms of performance. In this work, we find two main reasons for the weak performance: (1) Inaccurate evaluation setting. The evaluation setting under the closed-world assumption (CWA) may underestimate the PLM-based KGC models since they introduce more external knowledge; (2) Inappropriate utilization of PLMs. Most PLM-based KGC models simply splice the labels of entities and relations as inputs, leading to …

Go to article

Comai: Enabling Lightweight, Collaborative Intelligence By Retrofitting Vision Dnns, Kasthuri Jayarajah, Dhanuja Wanniarachchige, Tarek Abdelzaher, Archan Misra Apr 2022

Comai: Enabling Lightweight, Collaborative Intelligence By Retrofitting Vision Dnns, Kasthuri Jayarajah, Dhanuja Wanniarachchige, Tarek Abdelzaher, Archan Misra

Research Collection School Of Computing and Information Systems

While Deep Neural Network (DNN) models have transformed machine vision capabilities, their extremely high computational complexity and model sizes present a formidable deployment roadblock for AIoT applications. We show that the complexity-vs-accuracy-vs-communication tradeoffs for such DNN models can be significantly addressed via a novel, lightweight form of “collaborative machine intelligence” that requires only runtime changes to the inference process. In our proposed approach, called ComAI, the DNN pipelines of different vision sensors share intermediate processing state with one another, effectively providing hints about objects located within their mutually-overlapping Field-of-Views (FoVs). CoMAI uses two novel techniques: (a) a secondary shallow ML …

Go to article

Gesturelens: Visual Analysis Of Gestures In Presentation Videos, Haipeng Zeng, Xingbo Wang, Yong Wang, Aoyu Wu, Ting Chuen Pong, Huamin Qu Apr 2022

Gesturelens: Visual Analysis Of Gestures In Presentation Videos, Haipeng Zeng, Xingbo Wang, Yong Wang, Aoyu Wu, Ting Chuen Pong, Huamin Qu

Research Collection School Of Computing and Information Systems

Appropriate gestures can enhance message delivery and audience engagement in both daily communication and public presentations. In this paper, we contribute a visual analytic approach that assists professional public speaking coaches in improving their practice of gesture training through analyzing presentation videos. Manually checking and exploring gesture usage in the presentation videos is often tedious and time-consuming. There lacks an efficient method to help users conduct gesture exploration, which is challenging due to the intrinsically temporal evolution of gestures and their complex correlation to speech content. In this paper, we propose GestureLens, a visual analytics system to facilitate gesture-based and …

Go to article

On Size-Oriented Long-Tailed Graph Classification Of Graph Neural Networks, Zemin Liu, Qiheng Mao, Chenghao Liu, Yuan Fang, Jianling Sun Apr 2022

On Size-Oriented Long-Tailed Graph Classification Of Graph Neural Networks, Zemin Liu, Qiheng Mao, Chenghao Liu, Yuan Fang, Jianling Sun

Research Collection School Of Computing and Information Systems

The prevalence of graph structures attracts a surge of investigation on graph data, enabling several downstream tasks such as multigraph classification. However, in the multi-graph setting, graphs usually follow a long-tailed distribution in terms of their sizes, i.e., the number of nodes. In particular, a large fraction of tail graphs usually have small sizes. Though recent graph neural networks (GNNs) can learn powerful graph-level representations, they treat the graphs uniformly and marginalize the tail graphs which suffer from the lack of distinguishable structures, resulting in inferior performance on tail graphs. To alleviate this concern, in this paper we propose a …

Go to article

Immersivepov: Filming How-To Videos With A Head-Mounted 360° Action Camera, Kevin Huang, Jiannan Li, Maurício Sousa, Tovi Grossman Apr 2022

Immersivepov: Filming How-To Videos With A Head-Mounted 360° Action Camera, Kevin Huang, Jiannan Li, Maurício Sousa, Tovi Grossman

Research Collection School Of Computing and Information Systems

How-to videos are often shot using camera angles that may not be optimal for learning motor tasks, with a prevalent use of third-person perspective. We present immersivePOV, an approach to film how-to videos from an immersive first-person perspective using a head-mounted 360° action camera. immersivePOV how-to videos can be viewed in a Virtual Reality headset, giving the viewer an eye-level viewpoint with three Degrees of Freedom. We evaluated our approach with two everyday motor tasks against a baseline first-person perspective and a third-person perspective. In a between-subjects study, participants were assigned to watch the task videos and then replicate the …

Go to article

Rescuecastr: Exploring Photos And Live Streaming To Support Contextual Awareness In The Wilderness Search And Rescue Command Post, Brennon Jones, Anthony Tang, Carman Neustaedter Apr 2022

Rescuecastr: Exploring Photos And Live Streaming To Support Contextual Awareness In The Wilderness Search And Rescue Command Post, Brennon Jones, Anthony Tang, Carman Neustaedter

Research Collection School Of Computing and Information Systems

Wilderness search and rescue (WSAR) is a command-and-control activity where a Command team manages field teams scattered across a large area looking for a lost person. The challenge is that it can be difficult for Command to maintain awareness of field teams and the conditions of the field. We designed RescueCASTR, an interface that explores the idea of deploying field teams with wearable cameras that stream live video or sequential photos periodically to Command that aid contextual awareness. We ran a remote user study with WSAR managers to understand the opportunities and challenges of such a system. We found that …

Go to article

Trend: Temporal Event And Node Dynamics For Graph Representation Learning, Zhihao Wen, Yuan Fang Apr 2022

Trend: Temporal Event And Node Dynamics For Graph Representation Learning, Zhihao Wen, Yuan Fang

Research Collection School Of Computing and Information Systems

Temporal graph representation learning has drawn significant attention for the prevalence of temporal graphs in the real world. However, most existing works resort to taking discrete snapshots of the temporal graph, or are not inductive to deal with new nodes, or do not model the exciting effects which is the ability of events to influence the occurrence of another event. In this work, We propose TREND, a novel framework for temporal graph representation learning, driven by TempoRal Event and Node Dynamics and built upon a Hawkes process-based graph neural network (GNN). TREND presents a few major advantages: (1) it is …

Go to article

Mask-Guided Deformation Adaptive Network For Human Parsing, Aihua Mao, Yuan Liang, Jianbo Jiao, Yongtuo Liu, Shengfeng He Mar 2022

Mask-Guided Deformation Adaptive Network For Human Parsing, Aihua Mao, Yuan Liang, Jianbo Jiao, Yongtuo Liu, Shengfeng He

Research Collection School Of Computing and Information Systems

Due to the challenges of densely compacted body parts, nonrigid clothing items, and severe overlap in crowd scenes, human parsing needs to focus more on multilevel feature representations compared to general scene parsing tasks. Based on this observation, we propose to introduce the auxiliary task of human mask and edge detection to facilitate human parsing. Different from human parsing, which exploits the discriminative features of each category, human mask and edge detection emphasizes the boundaries of semantic parsing regions and the difference between foreground humans and background clutter, which benefits the parsing predictions of crowd scenes and small human parts. …

Go to article

Mrim: Enabling Mixed-Resolution Imaging For Low-Power Pervasive Vision Tasks, Jiyan Wu, Vithurson Subasharan, Tuan Tran, Archan Misra Mar 2022

Mrim: Enabling Mixed-Resolution Imaging For Low-Power Pervasive Vision Tasks, Jiyan Wu, Vithurson Subasharan, Tuan Tran, Archan Misra

Research Collection School Of Computing and Information Systems

While many pervasive computing applications increasingly utilize real-time context extracted from a vision sensing infrastructure, the high energy overhead of DNN-based vision sensing pipelines remains a challenge for sustainable in-the-wild deployment. One common approach to reducing such energy overheads is the capture and transmission of lower-resolution images to an edge node (where the DNN inferencing task is executed), but this results in an accuracy-vs-energy tradeoff, as the DNN inference accuracy typically degrades with a drop in resolution. In this work, we introduce MRIM, a simple but effective framework to tackle this tradeoff. Under MRIM, the vision sensor platform first executes …

Go to article

Debiasing Nlu Models Via Causal Intervention And Counterfactual Reasoning, Bing Tian, Yixin Cao, Yong Zhang, Chunxiao Xing Mar 2022

Debiasing Nlu Models Via Causal Intervention And Counterfactual Reasoning, Bing Tian, Yixin Cao, Yong Zhang, Chunxiao Xing

Research Collection School Of Computing and Information Systems

Recent studies have shown that strong Natural Language Understanding (NLU) models are prone to relying on annotation biases of the datasets as a shortcut, which goes against the underlying mechanisms of the task of interest. To reduce such biases, several recent works introduce debiasing methods to regularize the training process of targeted NLU models. In this paper, we provide a new perspective with causal inference to fnd out the bias. On the one hand, we show that there is an unobserved confounder for the natural language utterances and their respective classes, leading to spurious correlations from training data. To remove …

Go to article

First Prev Next Last