Open Access. Powered by Scholars. Published by Universities.®

Graphics and Human Computer Interfaces Commons™

Open Access. Powered by Scholars. Published by Universities.®

Discipline
Institution
Keyword
Publication Year
Publication
Publication Type
File Type

Articles 721 - 750 of 2422

Full-Text Articles in Graphics and Human Computer Interfaces

Multivariate Fairness For Paper Selection, Reem Alsaffar Dec 2022

Multivariate Fairness For Paper Selection, Reem Alsaffar

Graduate Theses and Dissertations

Peer review is the process by which publishers select the best publications for inclusion in a journal or a conference. Bias in the peer review process can impact which papers are selected for inclusion in conferences and journals. Although often implicit, race, gender and other demographics can prevent members of underrepresented groups from presenting at major conferences. To try to avoid bias, many conferences use a double-blind review process to increase fairness during reviewing. However, recent studies argue that the bias has not been removed completely. Our research focuses on developing fair algorithms that correct for these biases and select …

Go to article

Appearance-Preserved Portrait-To-Anime Translation Via Proxy-Guided Domain Adaptation, Wenpeng Xiao, Cheng Xu, Jiajie Mai, Xuemiao Xu, Yue Li, Chengze Li, Xueting Liu, Shengfeng He Dec 2022

Appearance-Preserved Portrait-To-Anime Translation Via Proxy-Guided Domain Adaptation, Wenpeng Xiao, Cheng Xu, Jiajie Mai, Xuemiao Xu, Yue Li, Chengze Li, Xueting Liu, Shengfeng He

Research Collection School Of Computing and Information Systems

Converting a human portrait to anime style is a desirable but challenging problem. Existing methods fail to resolve this problem due to the large inherent gap between two domains that cannot be overcome by a simple direct mapping. For this reason, these methods struggle to preserve the appearance features in the original photo. In this paper, we discover an intermediate domain, the coser portrait (portraits of humans costuming as anime characters), that helps bridge this gap. It alleviates the learning ambiguity and loosens the mapping difficulty in a progressive manner. Specifically, we start from learning the mapping between coser and …

Go to article

Segment-Wise Time-Varying Dynamic Bayesian Network With Graph Regularization, Xing Yang, Chen Zhang, Baihua Zheng Dec 2022

Segment-Wise Time-Varying Dynamic Bayesian Network With Graph Regularization, Xing Yang, Chen Zhang, Baihua Zheng

Research Collection School Of Computing and Information Systems

Time-varying dynamic Bayesian network (TVDBN) is essential for describing time-evolving directed conditional dependence structures in complex multivariate systems. In this article, we construct a TVDBN model, together with a score-based method for its structure learning. The model adopts a vector autoregressive (VAR) model to describe inter-slice and intra-slice relations between variables. By allowing VAR parameters to change segment-wisely over time, the time-varying dynamics of the network structure can be described. Furthermore, considering some external information can provide additional similarity information of variables. Graph Laplacian is further imposed to regularize similar nodes to have similar network structures. The regularized maximum a …

Go to article

Enabling The Human Perception Of A Working Camera In Web Conferences Via Its Movement, Anish Shrestha Nov 2022

Enabling The Human Perception Of A Working Camera In Web Conferences Via Its Movement, Anish Shrestha

LSU Master's Theses

In recent years, video conferencing has seen a significant increase in its usage due to the COVID-19 pandemic. When casting user’s video to other participants, the videoconference applications (e.g. Zoom, FaceTime, Skype, etc.) mainly leverage 1) webcam’s LED-light indicator, 2) user’s video feedback in the software and 3) the software’s video on/off icons to remind the user whether the camera is being used. However, these methods all impose the responsibility on the user itself to check the camera status, and there have been numerous cases reported when users expose their privacy inadvertently due to not realizing that their camera is …

Go to article

Vfirm: Verifiable Fine-Grained Encrypted Image Retrieval In Multi-Owner Multi-User Settings, Qiuyun Tong, Yinbin Miao, Lei Chen, Jian Weng, Kim-Kwang Raymond Choo, Ximeng Liu, Robert H. Deng Nov 2022

Vfirm: Verifiable Fine-Grained Encrypted Image Retrieval In Multi-Owner Multi-User Settings, Qiuyun Tong, Yinbin Miao, Lei Chen, Jian Weng, Kim-Kwang Raymond Choo, Ximeng Liu, Robert H. Deng

Research Collection School Of Computing and Information Systems

To ensure the security of images outsourced to the malicious cloud without affecting searchability on such outsourced (typically encrypted) images, one could use privacy-preserving Content-Based Image Retrieval (CBIR) primitive. However, conventional privacy-preserving CBIR schemes based on Searchable Symmetric Encryption (SSE) are not capable of supporting efficient fine-grained access control and result verification simultaneously. Therefore, in this article, we propose a Verifiable Fine-grained encrypted Image Retrieval scheme in the Multi-owner multi-user settings (VFIRM). VFIRM first utilizes a novel polynomial-based access strategy to provide efficient fine-grained access control. Then, it employs the dual secure kk-nearest neighbor technique to distribute distinct keys to …

Go to article

Graph Neural Network With Self-Attention And Multi-Task Learning For Credit Default Risk Prediction, Zihao Li, Xianzhi Wang, Lina Yao, Yakun Chen, Guandong Xu, Ee-Peng Lim Nov 2022

Graph Neural Network With Self-Attention And Multi-Task Learning For Credit Default Risk Prediction, Zihao Li, Xianzhi Wang, Lina Yao, Yakun Chen, Guandong Xu, Ee-Peng Lim

Research Collection School Of Computing and Information Systems

We propose a graph neural network with self-attention and multi-task learning (SaM-GNN) to leverage the advantages of deep learning for credit default risk prediction. Our approach incorporates two parallel tasks based on shared intermediate vectors for input vector reconstruction and credit default risk prediction, respectively. To better leverage supervised data, we use self-attention layers for feature representation of categorical and numeric data; we further link raw data into a graph and use a graph convolution module to aggregate similar information and cope with missing values during constructing intermediate vectors. Our method does not heavily rely on feature engineering work and …

Go to article

Cscw 2022 Chairs' Welcome, Gary Hsieh, Anthony Tang Nov 2022

Cscw 2022 Chairs' Welcome, Gary Hsieh, Anthony Tang

Research Collection School Of Computing and Information Systems

We are excited to bring you the 25th ACM Conference on Computer-Supported Cooperative Work and Social Computing – CSCW 2022 – virtually. We were initially poised to hold CSCW 2022 in Taiwan to help strengthen the CSCW community’s relationship with Asia, but the uncertainty around COVID-19 pandemic restrictions made this an impractical idea. Yet, as designers, we view challenges as learning opportunities and know learning opportunities lead to new ideas and approaches. From the online and virtual conferences experiences of the past few years, we considered questions like: what makes conferences interesting, what makes conferences worth attending, how can we …

Go to article

A Cognitive-Emotional Model From Mobile Short-Form Video Addiction To Intermittent Discontinuance: The Moderating Role Of Neutralization, Yafei Feng, Lifu Li, Anqi Zhao Nov 2022

A Cognitive-Emotional Model From Mobile Short-Form Video Addiction To Intermittent Discontinuance: The Moderating Role Of Neutralization, Yafei Feng, Lifu Li, Anqi Zhao

Research Collection School Of Computing and Information Systems

Discontinuance is considered very effective in the treatment of IT addiction. However, prior literature on the mechanism from IT addiction to discontinuance has reached inconsistent conclusions. Also, scholars cannot explain why some individuals do not make any changes after becoming addicted, even when they realize that their behavior is not in line with the norms. To fill the gaps, our research developed a cognitive-emotional model, with neutralization as a moderator, to understand the internal mechanism from mobile short-form video addiction to intermittent discontinuance. Based on the online survey of 493 Chinese mobile short-form video users, this study used partial least …

Go to article

Hapticpuppet: A Kinesthetic Mid-Air Multidirectional Force-Feedback Drone-Based Interface, Martin Feick, Anthony Tang, Antonio Kruger Nov 2022

Hapticpuppet: A Kinesthetic Mid-Air Multidirectional Force-Feedback Drone-Based Interface, Martin Feick, Anthony Tang, Antonio Kruger

Research Collection School Of Computing and Information Systems

Providing kinesthetic force-feedback for human-scale interactions is challenging due to the relatively large forces needed. Therefore, robotic actuators are predominantly used to deliver this kind of haptic feedback; however, they offer limited flexibility and spatial resolution. In this work, we introduce HapticPuppet, a drone-based force-feedback interface which can exert multidirectional forces onto the human body. This can be achieved by attaching strings to different parts of the human body such as fingers, hands or ankles, which can then be affixed to multiple coordinated drones - puppeteering the user. HapticPuppet opens up a wide range of potential applications in virtual, augmented …

Go to article

Tourgether360: Collaborative Exploration Of 360° Videos Using Pseudo-Spatial Navigation, Kartikaeya Kumar, Lev Poretski, Jianan Li, Anthony Tang Nov 2022

Tourgether360: Collaborative Exploration Of 360° Videos Using Pseudo-Spatial Navigation, Kartikaeya Kumar, Lev Poretski, Jianan Li, Anthony Tang

Research Collection School Of Computing and Information Systems

Collaborative exploration of 360 videos with contemporary interfaces is challenging because collaborators do not have awareness of one another's viewing activities. Tourgether360 enhances social exploration of 360° tour videos using a pseudo-spatial navigation technique that provides both an overhead "context" view of the environment as a minimap, as well as a shared pseudo-3D environment for exploring the video. Collaborators are embodied as avatars along a track depending on their position in the video timeline and can point and synchronize their playback. We evaluated the Tourgether360 concept through two studies: first, a comparative study with a simplified version of Tourgether360 with …

Go to article

Daot: Domain-Agnostically Aligned Optimal Transport For Domain-Adaptive Crowd Counting, Huilin Zhu, Jingling Yuan, Xian Zhong, Zhengwei Yang, Zheng Wang, Shengfeng He Nov 2022

Daot: Domain-Agnostically Aligned Optimal Transport For Domain-Adaptive Crowd Counting, Huilin Zhu, Jingling Yuan, Xian Zhong, Zhengwei Yang, Zheng Wang, Shengfeng He

Research Collection School Of Computing and Information Systems

Domain adaptation is commonly employed in crowd counting to bridge the domain gaps between different datasets. However, existing domain adaptation methods tend to focus on inter-dataset differences while overlooking the intra-differences within the same dataset, leading to additional learning ambiguities. These domain-agnostic factors,e.g., density, surveillance perspective, and scale, can cause significant in-domain variations, and the misalignment of these factors across domains can lead to a drop in performance in cross-domain crowd counting. To address this issue, we propose a Domain-agnostically Aligned Optimal Transport (DAOT) strategy that aligns domain-agnostic factors between domains. The DAOT consists of three steps. First, individual-level differences …

Go to article

Epic-Kitchens Visor Benchmark: Video Segmentations And Object Relations, Ahmad Ak Dar Khalil, Dandan Shan, Bin Zhu, Jian Ma, Amlan Kar, Richard Higgins, David Fouhey, Sanja Fidler, Dima Damen Nov 2022

Epic-Kitchens Visor Benchmark: Video Segmentations And Object Relations, Ahmad Ak Dar Khalil, Dandan Shan, Bin Zhu, Jian Ma, Amlan Kar, Richard Higgins, David Fouhey, Sanja Fidler, Dima Damen

Research Collection School Of Computing and Information Systems

We introduce VISOR, a new dataset of pixel annotations and a benchmark suite for segmenting hands and active objects in egocentric video. VISOR annotates videos from EPIC-KITCHENS, which comes with a new set of challenges not encountered in current video segmentation datasets. Specifically, we need to ensure both short- and long-term consistency of pixel-level annotations as objects undergo transformative interactions, e.g. an onion is peeled, diced and cooked - where we aim to obtain accurate pixel-level annotations of the peel, onion pieces, chopping board, knife, pan, as well as the acting hands. VISOR introduces an annotation pipeline, AI-powered in parts, …

Go to article

Ideating Xai: An Exploration Of User’S Mental Models Of An Ai-Driven Recruitment System Using A Design Thinking Approach, Helen Sheridan, Dympna O'Sullivan, Emma Murphy Oct 2022

Ideating Xai: An Exploration Of User’S Mental Models Of An Ai-Driven Recruitment System Using A Design Thinking Approach, Helen Sheridan, Dympna O'Sullivan, Emma Murphy

Conference Papers

Artificial Intelligence (AI) is playing an important role in society including how vital, often life changing decisions are made. For this reason, interest in Explainable Artificial Intelligence (XAI) has grown in recent years as a means of revealing the processes and operations contained within what is often described as a black box, an often-opaque system whose decisions are difficult to understand by the end user. This paper presents the results of a design thinking workshop with 20 participants (computer science and graphic design students) where we sought to investigate users' mental models when interacting with AI systems. Using two personas, …

Go to article

Supporting Prosocial Behaviour In Online Communities Through Social Media Affordances, Dominique Kelly, Liu Yifan, Alex Mayhew, Sarah Cornwell, Yimin Chen, Nicole Delellis, Victoria Rubin Oct 2022

Supporting Prosocial Behaviour In Online Communities Through Social Media Affordances, Dominique Kelly, Liu Yifan, Alex Mayhew, Sarah Cornwell, Yimin Chen, Nicole Delellis, Victoria Rubin

Data and Test Instruments

Affordances are action possibilities that emerge from the relationship between the properties of an object and an interacting agent’s capabilities. This poster examines how one type of affordance—anonymity—is enabled or constrained by six features of social media platforms. Our work is one step in a broader agenda to: (1) identify affordances that influence users’ behaviour in online communities; (2) outline the social media features that enable or constrain those affordances; and, (3) experimentally determine whether certain affordances, or combinations of affordances, support prosocial behaviour. Prosocial behaviour in information and communication technologies (ICTs) is broadly viewed here as benefiting other individuals …

Go to article

Dynamic Temporal Filtering In Video Models, Fuchen Long, Zhaofan Qiu, Yingwei Pan, Ting Yao, Chong-Wah Ngo, Tao Mei Oct 2022

Dynamic Temporal Filtering In Video Models, Fuchen Long, Zhaofan Qiu, Yingwei Pan, Ting Yao, Chong-Wah Ngo, Tao Mei

Research Collection School Of Computing and Information Systems

Video temporal dynamics is conventionally modeled with 3D spatial-temporal kernel or its factorized version comprised of 2D spatial kernel and 1D temporal kernel. The modeling power, nevertheless, is limited by the fixed window size and static weights of a kernel along the temporal dimension. The pre-determined kernel size severely limits the temporal receptive fields and the fixed weights treat each spatial location across frames equally, resulting in sub-optimal solution for longrange temporal modeling in natural scenes. In this paper, we present a new recipe of temporal feature learning, namely Dynamic Temporal Filter (DTF), that novelly performs spatial-aware temporal modeling in …

Go to article

Cvfnet: Real-Time 3d Object Detection By Learning Cross View Features, Jiaqi Gu, Zhiyu Xiang, Pan Zhao, Tingming Bai, Lingxuan Wang, Xijun Zhao, Zhiyuan Zhang Oct 2022

Cvfnet: Real-Time 3d Object Detection By Learning Cross View Features, Jiaqi Gu, Zhiyu Xiang, Pan Zhao, Tingming Bai, Lingxuan Wang, Xijun Zhao, Zhiyuan Zhang

Research Collection School Of Computing and Information Systems

In recent years 3D object detection from LiDAR point clouds has made great progress thanks to the development of deep learning technologies. Although voxel or point based methods are popular in 3D object detection, they usually involve time-consuming operations such as 3D convolutions on voxels or ball query among points, making the resulting network inappropriate for time critical applications. On the other hand, 2D view-based methods feature high computing efficiency while usually obtaining inferior performance than the voxel or point based methods. In this work, we present a real-time view-based single stage 3D object detector, namely CVFNet to fulfill this …

Go to article

Tgdm: Target Guided Dynamic Mixup For Cross-Domain Few-Shot Learning, Linhai Zhuo, Yuqian Fu, Jingjing Chen, Yixin Cao, Yu-Gang Jiang Oct 2022

Tgdm: Target Guided Dynamic Mixup For Cross-Domain Few-Shot Learning, Linhai Zhuo, Yuqian Fu, Jingjing Chen, Yixin Cao, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Given sufficient training data on the source domain, cross-domain few-shot learning (CD-FSL) aims at recognizing new classes with a small number of labeled examples on the target domain. The key to addressing CD-FSL is to narrow the domain gap and transferring knowledge of a network trained on the source domain to the target domain. To help knowledge transfer, this paper introduces an intermediate domain generated by mixing images in the source and the target domain. Specifically, to generate the optimal intermediate domain for different target data, we propose a novel target guided dynamic mixup (TGDM) framework that leverages the target …

Go to article

Long-Term Leap Attention, Short-Term Periodic Shift For Video Classification, Hao Zhang, Lechao Cheng, Yanbin Hao, Chong-Wah Ngo Oct 2022

Long-Term Leap Attention, Short-Term Periodic Shift For Video Classification, Hao Zhang, Lechao Cheng, Yanbin Hao, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Video transformer naturally incurs a heavier computation burden than a static vision transformer, as the former processes �� times longer sequence than the latter under the current attention of quadratic complexity (�� 2�� 2 ). The existing works treat the temporal axis as a simple extension of spatial axes, focusing on shortening the spatio-temporal sequence by either generic pooling or local windowing without utilizing temporal redundancy. However, videos naturally contain redundant information between neighboring frames; thereby, we could potentially suppress attention on visually similar frames in a dilated manner. Based on this hypothesis, we propose the LAPS, a long-term “Leap …

Go to article

Wave-Vit: Unifying Wavelet And Transformers For Visual Representation Learning, Ting Yao, Yingwei Pan, Yehao Li, Chong-Wah Ngo, Tao Mei Oct 2022

Wave-Vit: Unifying Wavelet And Transformers For Visual Representation Learning, Ting Yao, Yingwei Pan, Yehao Li, Chong-Wah Ngo, Tao Mei

Research Collection School Of Computing and Information Systems

Multi-scale Vision Transformer (ViT) has emerged as a powerful backbone for computer vision tasks, while the self-attention computation in Transformer scales quadratically w.r.t. the input patch number. Thus, existing solutions commonly employ down-sampling operations (e.g., average pooling) over keys/values to dramatically reduce the computational cost. In this work, we argue that such over-aggressive down-sampling design is not invertible and inevitably causes information dropping especially for high-frequency components in objects (e.g., texture details). Motivated by the wavelet theory, we construct a new Wavelet Vision Transformer (Wave-ViT) that formulates the invertible down-sampling with wavelet transforms and self-attention learning in a unified way. …

Go to article

On Mitigating Hard Clusters For Face Clustering, Yingjie Chen, Huasong Zhong, Chong Chen, Chen Shen, Jianqiang Huang, Tao Wang, Yun Liang, Qianru Sun Oct 2022

On Mitigating Hard Clusters For Face Clustering, Yingjie Chen, Huasong Zhong, Chong Chen, Chen Shen, Jianqiang Huang, Tao Wang, Yun Liang, Qianru Sun

Research Collection School Of Computing and Information Systems

Face clustering is a promising way to scale up face recognition systems using large-scale unlabeled face images. It remains challenging to identify small or sparse face image clusters that we call hard clusters, which is caused by the heterogeneity, i.e., high variations in size and sparsity, of the clusters. Consequently, the conventional way of using a uniform threshold (to identify clusters) often leads to a terrible misclassification for the samples that should belong to hard clusters. We tackle this problem by leveraging the neighborhood information of samples and inferring the cluster memberships (of samples) in a probabilistic way. We introduce …

Go to article

Locally Varying Distance Transform For Unsupervised Visual Anomaly Detection, Wen-Yan Lin, Zhonghang Liu, Siying Liu Oct 2022

Locally Varying Distance Transform For Unsupervised Visual Anomaly Detection, Wen-Yan Lin, Zhonghang Liu, Siying Liu

Research Collection School Of Computing and Information Systems

Unsupervised anomaly detection on image data is notoriously unstable. We believe this is because many classical anomaly detectors implicitly assume data is low dimensional. However, image data is always high dimensional. Images can be projected to a low dimensional embedding but such projections rely on global transformations that truncate minor variations. As anomalies are rare, the final embedding often lacks the key variations needed to distinguish anomalies from normal instances. This paper proposes a new embedding using a set of locally varying data projections, with each projection responsible for persevering the variations that distinguish a local cluster of instances from …

Go to article

Interactive Contrastive Learning For Self-Supervised Entity Alignment, Kaisheng Zeng, Zhenhao Dong, Lei Hou, Yixin Cao, Minghao Hu, Jifan Yu, Xin Lv, Lei Cao, Xin Wang, Haozhuang Liu, Yi Huang, Jing Wan, Juanzi Li Oct 2022

Interactive Contrastive Learning For Self-Supervised Entity Alignment, Kaisheng Zeng, Zhenhao Dong, Lei Hou, Yixin Cao, Minghao Hu, Jifan Yu, Xin Lv, Lei Cao, Xin Wang, Haozhuang Liu, Yi Huang, Jing Wan, Juanzi Li

Research Collection School Of Computing and Information Systems

Self-supervised entity alignment (EA) aims to link equivalent entities across different knowledge graphs (KGs) without the use of pre-aligned entity pairs. The current state-of-the-art (SOTA) selfsupervised EA approach draws inspiration from contrastive learning, originally designed in computer vision based on instance discrimination and contrastive loss, and suffers from two shortcomings. Firstly, it puts unidirectional emphasis on pushing sampled negative entities far away rather than pulling positively aligned pairs close, as is done in the well-established supervised EA. Secondly, it advocates the minimum information requirement for self-supervised EA, while we argue that self-described KG’s side information (e.g., entity name, relation name, …

Go to article

Investigating Accessibility Challenges And Opportunities For Users With Low Vision Disabilities In Customer-To-Customer (C2c) Marketplaces, Bektur Ryskeldiev, Kotaro Hara, Mariko Kobayashi, Koki Kusano Oct 2022

Investigating Accessibility Challenges And Opportunities For Users With Low Vision Disabilities In Customer-To-Customer (C2c) Marketplaces, Bektur Ryskeldiev, Kotaro Hara, Mariko Kobayashi, Koki Kusano

Research Collection School Of Computing and Information Systems

Inaccessible e-commerce websites and mobile applications exclude people with visual impairments (PVI) from online shopping. Customer-to-customer (C2C) marketplaces, a form of e-commerce where trading happens not between businesses and customers but between customers, could pose a unique set of challenges in the interactions that the platform brings about. Through online questionnaire and remote interviews, we investigate problems experienced by people with low vision disabilities in common C2C scenarios. Our study with low vision participants (N = 12) reveal both previously known general accessibility issues (e.g., web and mobile interface accessibility) and C2C specific accessibility issues (e.g., inability to confirm item …

Go to article

Equivariance And Invariance Inductive Bias For Learning From Insufficient Data, Tan Wang, Qianru Sun, Sugiri Pranata, Karlekar Jayashree, Hanwang Zhang Oct 2022

Equivariance And Invariance Inductive Bias For Learning From Insufficient Data, Tan Wang, Qianru Sun, Sugiri Pranata, Karlekar Jayashree, Hanwang Zhang

Research Collection School Of Computing and Information Systems

We are interested in learning robust models from insufficient data, without the need for any externally pre-trained model checkpoints. First, compared to sufficient data, we show why insufficient data renders the model more easily biased to the limited training environments that are usually different from testing. For example, if all the training "swan" samples are "white", the model may wrongly use the "white" environment to represent the intrinsic class "swan". Then, we justify that equivariance inductive bias can retain the class feature while invariance inductive bias can remove the environmental feature, leaving only the class feature that generalizes to any …

Go to article

Pixel-Wise Energy-Biased Abstention Learning For Anomaly Segmentation On Complex Urban Driving Scenes, Yu Tian, Yuyuan Liu, Guansong Pang, Fengbei Liu, Yuanhong Chen, Gustavo Carneiro Oct 2022

Pixel-Wise Energy-Biased Abstention Learning For Anomaly Segmentation On Complex Urban Driving Scenes, Yu Tian, Yuyuan Liu, Guansong Pang, Fengbei Liu, Yuanhong Chen, Gustavo Carneiro

Research Collection School Of Computing and Information Systems

State-of-the-art (SOTA) anomaly segmentation approaches on complex urban driving scenes explore pixel-wise classification uncertainty learned from outlier exposure, or external reconstruction models. However, previous uncertainty approaches that directly associate high uncertainty to anomaly may sometimes lead to incorrect anomaly predictions, and external reconstruction models tend to be too inefficient for real-time self-driving embedded systems. In this paper, we propose a new anomaly segmentation method, named pixel-wise energy-biased abstention learning (PEBAL), that explores pixel-wise abstention learning (AL) with a model that learns an adaptive pixel-level anomaly class, and an energy-based model (EBM) that learns inlier pixel distribution. More specifically, PEBAL is …

Go to article

Learning Discriminative Representations Via Variational Self-Distillation For Cross-View Geo-Localization, Qian Hu, Wansi Li, Xing Xu, Ning Liu, Lei Wang Oct 2022

Learning Discriminative Representations Via Variational Self-Distillation For Cross-View Geo-Localization, Qian Hu, Wansi Li, Xing Xu, Ning Liu, Lei Wang

Research Collection School Of Computing and Information Systems

Cross-view geo-localization is to localize the same geographic target in images from different perspectives, e.g., satellite-view and drone-view. The primary challenge faced by existing methods is the large visual appearance changes across views. Most previous work utilizes the deep neural network to obtain the discriminative representations and directly uses them to accomplish the geo-localization task. However, these approaches ignore that the redundancy retained in the extracted features negatively impacts the result. In this paper, we argue that the information bottleneck (IB) can retain the most relevant information while removing as much redundancy as possible. The variational self-distillation (VSD) strategy provides …

Go to article

Unsupervised Video Hashing With Multi-Granularity Contextualization And Multi-Structure Preservation, Yanbin Hao, Jingru Duan, Hao Zhang, Bin Zhu, Pengyuan Zhou, Xiangnan He Oct 2022

Unsupervised Video Hashing With Multi-Granularity Contextualization And Multi-Structure Preservation, Yanbin Hao, Jingru Duan, Hao Zhang, Bin Zhu, Pengyuan Zhou, Xiangnan He

Research Collection School Of Computing and Information Systems

Unsupervised video hashing typically aims to learn a compact binary vector to represent complex video content without using manual annotations. Existing unsupervised hashing methods generally suffer from incomplete exploration of various perspective dependencies (e.g., long-range and short-range) and data structures that exist in visual contents, resulting in less discriminative hash codes. In this paper, we propose aMulti-granularity Contextualized and Multi-Structure preserved Hashing (MCMSH) method, exploring multiple axial contexts for discriminative video representation generation and various structural information for unsupervised learning simultaneously. Specifically, we delicately design three self-gating modules to separately model three granularities of dependencies (i.e., long/middle/short-range dependencies) and densely …

Go to article

Dualformer: Local-Global Stratified Transformer For Efficient Video Recognition, Yuxuan Liang, Pan Zhou, Roger Zimmermann, Shuicheng Yan Oct 2022

Dualformer: Local-Global Stratified Transformer For Efficient Video Recognition, Yuxuan Liang, Pan Zhou, Roger Zimmermann, Shuicheng Yan

Research Collection School Of Computing and Information Systems

While transformers have shown great potential on video recognition with their strong capability of capturing long-range dependencies, they often suffer high computational costs induced by the self-attention to the huge number of 3D tokens. In this paper, we present a new transformer architecture termed DualFormer, which can efficiently perform space-time attention for video recognition. Concretely, DualFormer stratifies the full space-time attention into dual cascaded levels, i.e., to first learn fine-grained local interactions among nearby 3D tokens, and then to capture coarse-grained global dependencies between the query token and global pyramid contexts. Different from existing methods that apply space-time factorization or …

Go to article

Self-Promoted Supervision For Few-Shot Transformer, Bowen Dong, Pan Zhou, Shuicheng Yan, Wangmeng Zuo Oct 2022

Self-Promoted Supervision For Few-Shot Transformer, Bowen Dong, Pan Zhou, Shuicheng Yan, Wangmeng Zuo

Research Collection School Of Computing and Information Systems

The few-shot learning ability of vision transformers (ViTs) is rarely investigated though heavily desired. In this work, we empirically find that with the same few-shot learning frameworks, e.g. MetaBaseline, replacing the widely used CNN feature extractor with a ViT model often severely impairs few-shot classification performance. Moreover, our empirical study shows that in the absence of inductive bias, ViTs often learn the low-qualified token dependencies under few-shot learning regime where only a few labeled training data are available, which largely contributes to the above performance degradation. To alleviate this issue, for the first time, we propose a simple yet effective …

Go to article

Video Graph Transformer For Video Question Answering, Junbin Xiao, Pan Zhou, Tat-Seng Chua, Shuicheng Yan Oct 2022

Video Graph Transformer For Video Question Answering, Junbin Xiao, Pan Zhou, Tat-Seng Chua, Shuicheng Yan

Research Collection School Of Computing and Information Systems

This paper proposes a Video Graph Transformer (VGT) model for Video Quetion Answering (VideoQA). VGT’s uniqueness are two-fold: 1) it designs a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations, and dynamics for complex spatio-temporal reasoning; and 2) it exploits disentangled video and text Transformers for relevance comparison between the video and text to perform QA, instead of entangled crossmodal Transformer for answer classification. Vision-text communication is done by additional cross-modal interaction modules. With more reasonable video encoding and QA solution, we show that VGT can achieve much better performances on VideoQA tasks …

Go to article

First Prev Next Last