Open Access. Powered by Scholars. Published by Universities.®

Graphics and Human Computer Interfaces Commons™

Open Access. Powered by Scholars. Published by Universities.®

Singapore Management University

Discipline
Keyword
Publication Year
Publication
Publication Type

Articles 271 - 300 of 959

Full-Text Articles in Graphics and Human Computer Interfaces

Turn-It-Up: Rendering Resistance For Knobs In Virtual Reality Through Undetectable Pseudo-Haptics, Martin Feick, Andre Zenner, Oscar Ariza, Anthony Tang, Cihan Biyikli, Antonio Kruger Nov 2023

Turn-It-Up: Rendering Resistance For Knobs In Virtual Reality Through Undetectable Pseudo-Haptics, Martin Feick, Andre Zenner, Oscar Ariza, Anthony Tang, Cihan Biyikli, Antonio Kruger

Research Collection School Of Computing and Information Systems

Rendering haptic feedback for interactions with virtual objects is an essential part of effective virtual reality experiences. In this work, we explore providing haptic feedback for rotational manipulations, e.g., through knobs. We propose the use of a Pseudo-Haptic technique alongside a physical proxy knob to simulate various physical resistances. In a psychophysical experiment with 20 participants, we found that designers can introduce unnoticeable offsets between real and virtual rotations of the knob, and we report the corresponding detection thresholds. Based on these, we present the Pseudo-Haptic Resistance technique to convey physical resistance while applying only unnoticeable pseudo-haptic manipulation. Additionally, we …

Go to article

Voxelhap: A Toolkit For Constructing Proxies Providing Tactile And Kinesthetic Haptic Feedback In Virtual Reality, M. Feick, C. Biyikli, K. Gani, A. Wittig, Anthony Tang, A. Krüger Nov 2023

Voxelhap: A Toolkit For Constructing Proxies Providing Tactile And Kinesthetic Haptic Feedback In Virtual Reality, M. Feick, C. Biyikli, K. Gani, A. Wittig, Anthony Tang, A. Krüger

Research Collection School Of Computing and Information Systems

Experiencing virtual environments is often limited to abstract interactions with objects. Physical proxies allow users to feel virtual objects, but are often inaccessible. We present the VoxelHap toolkit which enables users to construct highly functional proxy objects using Voxels and Plates. Voxels are blocks with special functionalities that form the core of each physical proxy. Plates increase a proxy’s haptic resolution, such as its shape, texture or weight. Beyond providing physical capabilities to realize haptic sensations, VoxelHap utilizes VR illusion techniques to expand its haptic resolution. We evaluated the capabilities of the VoxelHap toolkit through the construction of a range …

Go to article

Npf-200: A Multi-Modal Eye Fixation Dataset And Method For Non-Photorealistic Videos, Ziyu Yang, Sucheng Ren, Zongwei Wu, Nanxuan Zhao, Junle Wang, Jing Qin, Shengfeng He Nov 2023

Npf-200: A Multi-Modal Eye Fixation Dataset And Method For Non-Photorealistic Videos, Ziyu Yang, Sucheng Ren, Zongwei Wu, Nanxuan Zhao, Junle Wang, Jing Qin, Shengfeng He

Research Collection School Of Computing and Information Systems

Non-photorealistic videos are in demand with the wave of the metaverse, but lack of sufficient research studies. This work aims to take a step forward to understand how humans perceive nonphotorealistic videos with eye fixation (i.e., saliency detection), which is critical for enhancing media production, artistic design, and game user experience. To fill in the gap of missing a suitable dataset for this research line, we present NPF-200, the first largescale multi-modal dataset of purely non-photorealistic videos with eye fixations. Our dataset has three characteristics: 1) it contains soundtracks that are essential according to vision and psychological studies; 2) it …

Go to article

Matk: The Meme Analytical Tool Kit, Ming Shan Hee, Aditi Kumaresan, Nguyen Khoi Hoang, Nirmalendu Prakash, Rui Cao, Roy Ka-Wei Lee Nov 2023

Matk: The Meme Analytical Tool Kit, Ming Shan Hee, Aditi Kumaresan, Nguyen Khoi Hoang, Nirmalendu Prakash, Rui Cao, Roy Ka-Wei Lee

Research Collection School Of Computing and Information Systems

The rise of social media platforms has brought about a new digital culture called memes. Memes, which combine visuals and text, can strongly influence public opinions on social and cultural issues. As a result, people have become interested in categorizing memes, leading to the development of various datasets and multimodal models that show promising results in this field. However, there is currently a lack of a single library that allows for the reproduction, evaluation, and comparison of these models using fair benchmarks and settings. To fill this gap, we introduce the Meme Analytical Tool Kit (MATK), an open-source toolkit specifically …

Go to article

Editanything: Empowering Unparalleled Flexibility In Image Editing And Generation, Shanghua Gao, Zhijie Lin, Xingyu Xie, Pan Zhou, Ming-Ming Cheng, Shuicheng Yan Nov 2023

Editanything: Empowering Unparalleled Flexibility In Image Editing And Generation, Shanghua Gao, Zhijie Lin, Xingyu Xie, Pan Zhou, Ming-Ming Cheng, Shuicheng Yan

Research Collection School Of Computing and Information Systems

Image editing plays a vital role in computer vision field, aiming to realistically manipulate images while ensuring seamless integration. It finds numerous applications across various fields. In this work, we present EditAnything, a novel approach that empowers users with unparalleled flexibility in editing and generating image content. EditAnything introduces an array of advanced features, including crossimage dragging (e.g., try-on), region-interactive editing, controllable layout generation, and virtual character replacement. By harnessing these capabilities, users can engage in interactive and flexible editing, giving captivating outcomes that uphold the integrity of the original image. With its diverse range of tools, EditAnything caters to …

Go to article

Demo Abstract: Vgglass - Demonstrating Visual Grounding And Localization Synergy With A Lidar-Enabled Smart-Glass, Darshana Rathnayake, Dulanga Weerakoon, Meeralakshmi Radhakrishnan, Vigneshwaran Subbaraju, Inseok Hwang, Archan Misra Nov 2023

Demo Abstract: Vgglass - Demonstrating Visual Grounding And Localization Synergy With A Lidar-Enabled Smart-Glass, Darshana Rathnayake, Dulanga Weerakoon, Meeralakshmi Radhakrishnan, Vigneshwaran Subbaraju, Inseok Hwang, Archan Misra

Research Collection School Of Computing and Information Systems

This work demonstrates the VGGlass system, which simultaneously interprets human instructions for a target acquisition task and determines the precise 3D positions of both user and the target object. This is achieved by utilizing LiDARs mounted in the infrastructure and a smart glass device worn by the user. Key to our system is the union of LiDAR-based localization termed LiLOC and a multi-modal visual grounding approach termed RealG(2)In-Lite. To demonstrate the system, we use Intel RealSense L515 cameras and a Microsoft HoloLens 2, as the user devices. VGGlass is able to: a) track the user in real-time in a global …

Go to article

Rigid: Recurrent Gan Inversion And Editing Of Real Face Videos, Yangyang Xu, Shengfeng He, Kwan-Yee K. Wong, Pingluo Luo Oct 2023

Rigid: Recurrent Gan Inversion And Editing Of Real Face Videos, Yangyang Xu, Shengfeng He, Kwan-Yee K. Wong, Pingluo Luo

Research Collection School Of Computing and Information Systems

GAN inversion is indispensable for applying the powerful editability of GAN to real images. However, existing methods invert video frames individually often leading to undesired inconsistent results over time. In this paper, we propose a unified recurrent framework, named Recurrent vIdeo GAN Inversion and eDiting (RIGID), to explicitly and simultaneously enforce temporally coherent GAN inversion and facial editing of real videos. Our approach models the temporal relations between current and previous frames from three aspects. To enable a faithful real video reconstruction, we first maximize the inversion fidelity and consistency by learning a temporal compensated latent code. Second, we observe …

Go to article

Unsupervised Anomaly Detection In Medical Images With A Memory-Augmented Multi-Level Cross-Attentional Masked Autoencoder, Yu Tian, Guansong Pang, Yuyuan Liu, Chong Wang, Yuanhong Chen, Fengbei Liu, Rajvinder Singh, Johan W. Verjans, Mengyu Wang, Gustavo Carneiro Oct 2023

Unsupervised Anomaly Detection In Medical Images With A Memory-Augmented Multi-Level Cross-Attentional Masked Autoencoder, Yu Tian, Guansong Pang, Yuyuan Liu, Chong Wang, Yuanhong Chen, Fengbei Liu, Rajvinder Singh, Johan W. Verjans, Mengyu Wang, Gustavo Carneiro

Research Collection School Of Computing and Information Systems

Unsupervised anomaly detection (UAD) aims to find anomalous images by optimising a detector using a training set that contains only normal images. UAD approaches can be based on reconstruction methods, self-supervised approaches, and Imagenet pre-trained models. Reconstruction methods, which detect anomalies from image reconstruction errors, are advantageous because they do not rely on the design of problem-specific pretext tasks needed by self-supervised approaches, and on the unreliable translation of models pre-trained from non-medical datasets. However, reconstruction methods may fail because they can have low reconstruction errors even for anomalous images. In this paper, we introduce a new reconstruction-based UAD approach …

Go to article

Diffuse3d: Wide-Angle 3d Photography Via Bilateral Diffusion, Yutao Jiang, Yang Zhou, Yuan Liang, Wenxi Liu, Jianbo Jiao, Yuhui Quan, Shengfeng He Oct 2023

Diffuse3d: Wide-Angle 3d Photography Via Bilateral Diffusion, Yutao Jiang, Yang Zhou, Yuan Liang, Wenxi Liu, Jianbo Jiao, Yuhui Quan, Shengfeng He

Research Collection School Of Computing and Information Systems

This paper aims to resolve the challenging problem of wide-angle novel view synthesis from a single image, a.k.a. wide-angle 3D photography. Existing approaches rely on local context and treat them equally to inpaint occluded RGB and depth regions, which fail to deal with large-region occlusion (i.e., observing from an extreme angle) and foreground layers might blend into background inpainting. To address the above issues, we propose Diffuse3D which employs a pre-trained diffusion model for global synthesis, while amending the model to activate depth-aware inference. Our key insight is to alter the convolution mechanism in the denoising process. We inject depth …

Go to article

Stprivacy: Spatio-Temporal Privacy-Preserving Action Recognition, Ming Li, Xiangyu Xu, Hehe Fan, Pan Zhou, Jun Liu, Jia-Wei Liu, Jiahe Li, Jussi Keppo, Mike Zheng Shou, Shuicheng Yan Oct 2023

Stprivacy: Spatio-Temporal Privacy-Preserving Action Recognition, Ming Li, Xiangyu Xu, Hehe Fan, Pan Zhou, Jun Liu, Jia-Wei Liu, Jiahe Li, Jussi Keppo, Mike Zheng Shou, Shuicheng Yan

Research Collection School Of Computing and Information Systems

Existing methods of privacy-preserving action recognition (PPAR) mainly focus on frame-level (spatial) privacy removal through 2D CNNs. Unfortunately, they have two major drawbacks. First, they may compromise temporal dynamics in input videos, which are critical for accurate action recognition. Second, they are vulnerable to practical attacking scenarios where attackers probe for privacy from an entire video rather than individual frames. To address these issues, we propose a novel framework STPrivacy to perform video-level PPAR. For the first time, we introduce vision Transformers into PPAR by treating a video as a tubelet sequence, and accordingly design two complementary mechanisms, i.e., sparsification …

Go to article

Ubisurface: A Robotic Touch Surface For Supporting Mid-Air Planar Interactions In Room-Scale Vr, Ryota Gomi, Kazuki Takashima, Yuki Onishi, Kazuyuki Fujita, Yoshifumi Kitamura Oct 2023

Ubisurface: A Robotic Touch Surface For Supporting Mid-Air Planar Interactions In Room-Scale Vr, Ryota Gomi, Kazuki Takashima, Yuki Onishi, Kazuyuki Fujita, Yoshifumi Kitamura

Research Collection School Of Computing and Information Systems

Room-scale VR has been considered an alternative to physical office workspaces. For office activities, users frequently require planar input methods, such as typing or handwriting, to quickly record annotations to virtual content. However, current off-The-shelf VR HMD setups rely on mid-Air interactions, which can cause arm fatigue and decrease input accuracy. To address this issue, we propose UbiSurface, a robotic touch surface that can automatically reposition itself to physically present a virtual planar input surface (VR whiteboard, VR canvas, etc.) to users and to permit them to achieve accurate and fatigue-less input while walking around a virtual room. We design …

Go to article

Ciri: Curricular Inactivation For Residue-Aware One-Shot Video Inpainting, Weiying Zheng, Cheng Xu, Xuemiao Xu, Wenxi Liu, Shengfeng He Oct 2023

Ciri: Curricular Inactivation For Residue-Aware One-Shot Video Inpainting, Weiying Zheng, Cheng Xu, Xuemiao Xu, Wenxi Liu, Shengfeng He

Research Collection School Of Computing and Information Systems

Video inpainting aims at filling in missing regions of a video. However, when dealing with dynamic scenes with camera or object movements, annotating the inpainting target becomes laborious and impractical. In this paper, we resolve the one-shot video inpainting problem in which only one annotated first frame is provided. A naive solution is to propagate the initial target to the other frames with techniques like object tracking. In this context, the main obstacles are the unreliable propagation and the partially inpainted artifacts due to the inaccurate mask. For the former problem, we propose curricular inactivation to replace the hard masking …

Go to article

Deep Video Demoireing Via Compact Invertible Dyadic Decomposition, Yuhui Quan, Haoran Huang, Shengfeng He, Ruotao Xu Oct 2023

Deep Video Demoireing Via Compact Invertible Dyadic Decomposition, Yuhui Quan, Haoran Huang, Shengfeng He, Ruotao Xu

Research Collection School Of Computing and Information Systems

Removing moire patterns from videos recorded on screens or complex textures is known as video demoireing. It is a challenging task as both structures and textures of an image usually exhibit strong periodic patterns, which thus are easily confused with moire patterns and can be significantly erased in the removal process. By interpreting video demoireing as a multi-frame decomposition problem, we propose a compact invertible dyadic network called CIDNet that progressively decouples latent frames and the moire patterns from an input video sequence. Using a dyadic cross-scale coupling structure with coupling layers tailored for multi-scale processing, CIDNet aims at disentangling …

Go to article

Experiences Of Autistic Twitch Livestreamers: “I Have Made Easily The Most Meaningful And Impactful Relationships”, Terrance Mok, Anthony Tang, Adam Mccrimmon, Lora Oehlberg Oct 2023

Experiences Of Autistic Twitch Livestreamers: “I Have Made Easily The Most Meaningful And Impactful Relationships”, Terrance Mok, Anthony Tang, Adam Mccrimmon, Lora Oehlberg

Research Collection School Of Computing and Information Systems

We present perspectives from 10 autistic Twitch streamers regarding their experiences as livestreamers and how autism uniquely colors their experiences. Livestreaming offers a social online experience distinct from in-person, face-to-face communication, where autistic people tend to encounter challenges. Our reflexive thematic analysis of interviews with 10 participants showcases autistic livestreamers’ perspectives in their own words. Our findings center on the importance of having streamers establishing connections with other, sharing autistic identities, controlling a space for social interaction, personal growth, and accessibility challenges. In our discussion, we highlight the crucial value of having a medium for autistic representation, as well as …

Go to article

Underwater Image Translation Via Multi-Scale Generative Adversarial Network, Dongmei Yang, Tianzi Zhang, Boquan Li, Menghao Li, Weijing Chen, Xiaoqing Li, Xingmei Wang Oct 2023

Underwater Image Translation Via Multi-Scale Generative Adversarial Network, Dongmei Yang, Tianzi Zhang, Boquan Li, Menghao Li, Weijing Chen, Xiaoqing Li, Xingmei Wang

Research Collection School Of Computing and Information Systems

The role that underwater image translation plays assists in generating rare images for marine applications. However, such translation tasks are still challenging due to data lacking, insufficient feature extraction ability, and the loss of content details. To address these issues, we propose a novel multi-scale image translation model based on style-independent discriminators and attention modules (SID-AM-MSITM), which learns the mapping relationship between two unpaired images for translation. We introduce Convolution Block Attention Modules (CBAM) to the generators and discriminators of SID-AM-MSITM to improve its feature extraction ability. Moreover, we construct style-independent discriminators that enable the discriminant results of SID-AM-MSITM to …

Go to article

Feature Prediction Diffusion Model For Video Anomaly Detection, Cheng Yan, Shiyu Zhang, Yang Liu, Guansong Pang, Wenjun Wang Oct 2023

Feature Prediction Diffusion Model For Video Anomaly Detection, Cheng Yan, Shiyu Zhang, Yang Liu, Guansong Pang, Wenjun Wang

Research Collection School Of Computing and Information Systems

Anomaly detection in the video is an important research area and a challenging task in real applications. Due to the unavailability of large-scale annotated anomaly events, most existing video anomaly detection (VAD) methods focus on learning the distribution of normal samples to detect the substantially deviated samples as anomalies. To well learn the distribution of normal motion and appearance, many auxiliary networks are employed to extract foreground object or action information. These high-level semantic features effectively filter the noise from the background to decrease its influence on detection models. However, the capability of these extra semantic models heavily affects the …

Go to article

Supporting Artefact Awareness In Partially-Replicated Workspaces, Emran Poh, Anthony Tang, Jenanie S. Lee, Zhao Shengdong Oct 2023

Supporting Artefact Awareness In Partially-Replicated Workspaces, Emran Poh, Anthony Tang, Jenanie S. Lee, Zhao Shengdong

Research Collection School Of Computing and Information Systems

Using Cross Reality (CR) approaches for remote collaboration will often result in partially-replicated workspaces. Here, workspace artefacts are not equally accessible - i.e. a physical artefact may only be manipulated by one collaborator - and in general, the artefacts become desynchronised over time. In this paper, we introduce a framework for artefact awareness that can help collaborators maintain an understanding of each others' manipulations with workspace artefacts. We illustrate our design explorations through sketches, and outline how we aim to study the effectiveness and utility of artefact awareness in cross reality remote collaboration. In our work, we expect to show …

Go to article

Masked Diffusion Transformer Is A Strong Image Synthesizer, Shanghua Gao, Pan Zhou, Ming-Ming Cheng, Shuicheng Yan Oct 2023

Masked Diffusion Transformer Is A Strong Image Synthesizer, Shanghua Gao, Pan Zhou, Ming-Ming Cheng, Shuicheng Yan

Research Collection School Of Computing and Information Systems

Despite its success in image synthesis, we observe that diffusion probabilistic models (DPMs) often lack contextual reasoning ability to learn the relations among object parts in an image, leading to a slow learning process. To solve this issue, we propose a Masked Diffusion Transformer (MDT) that introduces a mask latent modeling scheme to explicitly enhance the DPMs’ ability to contextual relation learning among object semantic parts in an image. During training, MDT operates in the latent space to mask certain tokens. Then, an asymmetric masking diffusion transformer is designed to predict masked tokens from unmasked ones while maintaining the diffusion …

Go to article

Edge Distraction-Aware Salient Object Detection, Sucheng Ren, Wenxi Liu, Jianbo Jiao, Guoqiang Han, Shengfeng He Sep 2023

Edge Distraction-Aware Salient Object Detection, Sucheng Ren, Wenxi Liu, Jianbo Jiao, Guoqiang Han, Shengfeng He

Research Collection School Of Computing and Information Systems

Integrating low-level edge features has been proven to be effective in preserving clear boundaries of salient objects. However, the locality of edge features makes it difficult to capture globally salient edges, leading to distraction in the final predictions. To address this problem, we propose to produce distraction-free edge features by incorporating cross-scale holistic interdependencies between high-level features. In particular, we first formulate our edge features extraction process as a boundary-filling problem. In this way, we enforce edge features to focus on closed boundaries instead of those disconnected background edges. Second, we propose to explore cross-scale holistic contextual connections between every …

Go to article

Graph-Level Anomaly Detection Via Hierarchical Memory Networks, Chaoxi Niu, Guansong Pang, Ling Chen Sep 2023

Graph-Level Anomaly Detection Via Hierarchical Memory Networks, Chaoxi Niu, Guansong Pang, Ling Chen

Research Collection School Of Computing and Information Systems

Graph-level anomaly detection aims to identify abnormal graphs that exhibit deviant structures and node attributes compared to the majority in a graph set. One primary challenge is to learn normal patterns manifested in both fine-grained and holistic views of graphs for identifying graphs that are abnormal in part or in whole. To tackle this challenge, we propose a novel approach called Hierarchical Memory Networks (HimNet), which learns hierarchical memory modules---node and graph memory modules---via a graph autoencoder network architecture. The node-level memory module is trained to model fine-grained, internal graph interactions among nodes for detecting locally abnormal graphs, while the …

Go to article

Hyperbolic Graph Topic Modeling Network With Continuously Updated Topic Tree, Ce Zhang, Rex Ying, Hady Wirawan Lauw Aug 2023

Hyperbolic Graph Topic Modeling Network With Continuously Updated Topic Tree, Ce Zhang, Rex Ying, Hady Wirawan Lauw

Research Collection School Of Computing and Information Systems

Connectivity across documents often exhibits a hierarchical network structure. Hyperbolic Graph Neural Networks (HGNNs) have shown promise in preserving network hierarchy. However, they do not model the notion of topics, thus document representations lack semantic interpretability. On the other hand, a corpus of documents usually has high variability in degrees of topic specificity. For example, some documents contain general content (e.g., sports), while others focus on specific themes (e.g., basketball and swimming). Topic models indeed model latent topics for semantic interpretability, but most assume a flat topic structure and ignore such semantic hierarchy. Given these two challenges, we propose a …

Go to article

Flood: A Flexible Invariant Learning Framework For Out-Of-Distribution Generalization On Graphs, Yang Liu, Xiang Ao, Fuli Feng, Yunshan Ma, Kuan Li, Tat‑Seng Chua, Qing He Aug 2023

Flood: A Flexible Invariant Learning Framework For Out-Of-Distribution Generalization On Graphs, Yang Liu, Xiang Ao, Fuli Feng, Yunshan Ma, Kuan Li, Tat‑Seng Chua, Qing He

Research Collection School Of Computing and Information Systems

Graph Neural Networks (GNNs) have achieved remarkable success in various domains but most of them are developed under the in-distribution assumption. Under out-of-distribution (OOD) settings, they suffer from the distribution shift between the training set and the test set and may not generalize well to the test distribution. Several methods have tried the invariance principle to improve the generalization of GNNs in OOD settings. However, in previous solutions, the graph encoder is immutable after the invariant learning and cannot be adapted to the target distribution flexibly. Confronting the distribution shift, a flexible encoder with refinement to the target distribution can …

Go to article

Fine-Grained Domain Adaptive Crowd Counting Via Point-Derived Segmentation, Yongtuo Liu, Dan Xu, Sucheng Ren, Hanjie Wu, Hongmin Cai, Shengfeng He Jul 2023

Fine-Grained Domain Adaptive Crowd Counting Via Point-Derived Segmentation, Yongtuo Liu, Dan Xu, Sucheng Ren, Hanjie Wu, Hongmin Cai, Shengfeng He

Research Collection School Of Computing and Information Systems

Due to domain shift, a large performance drop is usually observed when a trained crowd counting model is deployed in the wild. While existing domain-adaptive crowd counting methods achieve promising results, they typically regard each crowd image as a whole and reduce domain discrepancies in a holistic manner, thus limiting further improvement of domain adaptation performance. To this end, we propose to untangle domain-invariant crowd and domain-specific background from crowd images and design a fine-grained domain adaption method for crowd counting. Specifically, to disentangle crowd from background, we propose to learn crowd segmentation from point-level crowd counting annotations in a …

Go to article

A Review On The Effects Of Chanting And Solfeggio Frequencies On Well-Being, Xuyu Yang, Fiona Fui-Hoon Nah, Fen Lin Jul 2023

A Review On The Effects Of Chanting And Solfeggio Frequencies On Well-Being, Xuyu Yang, Fiona Fui-Hoon Nah, Fen Lin

Research Collection School Of Computing and Information Systems

This paper presents our literature review on how chanting and solfeggio frequencies affect brain activity, enhance well-being, and inform the design of sound therapy. We call for more scientific research to investigate and better understand the effects of chanting and solfeggio frequencies on well-being.

Go to article

Socialz: Multi-Feature Social Fuzz Testing, Francisco Zanartu, Christoph Treude, Markus Wagner Jul 2023

Socialz: Multi-Feature Social Fuzz Testing, Francisco Zanartu, Christoph Treude, Markus Wagner

Research Collection School Of Computing and Information Systems

Online social networks have become an integral aspect of our daily lives and play a crucial role in shaping our relationships with others. However, bugs and glitches, even minor ones, can cause anything from frustrating problems to serious data leaks that can have farreaching impacts on millions of users. To mitigate these risks, fuzz testing, a method of testing with randomised inputs, can provide increased confidence in the correct functioning of a social network. However, implementing traditional fuzz testing methods can be prohibitively difficult or impractical for programmers outside of the network’s development team. To tackle this challenge, we present …

Go to article

Contrastive Video Question Answering Via Video Graph Transformer, Junbin Xiao Xiao, Pan Zhou, Angela Yao, Yicong Li, Richang Hong, Shuicheng Yan, Tat-Seng Chua Jul 2023

Contrastive Video Question Answering Via Video Graph Transformer, Junbin Xiao Xiao, Pan Zhou, Angela Yao, Yicong Li, Richang Hong, Shuicheng Yan, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

We propose to perform video question answering (VideoQA) in a Contrastive manner via a Video Graph Transformer model (CoVGT). CoVGT’s uniqueness and superiority are three-fold: 1) It proposes a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations and dynamics, for complex spatio-temporal reasoning. 2) It designs separate video and text transformers for contrastive learning between the video and text to perform QA, instead of multi-modal transformer for answer classification. Fine-grained video-text communication is done by additional cross-modal interaction modules. 3) It is optimized by the joint fully- and self-supervised contrastive objectives between the …

Go to article

Scanet: Self-Paced Semi-Curricular Attention Network For Non-Homogeneous Image Dehazing, Yu Guo, Yuan Gao, Ryan Wen Liu, Yuxu Lu, Jingxiang Qu, Shengfeng He, Ren Wenqi Jun 2023

Scanet: Self-Paced Semi-Curricular Attention Network For Non-Homogeneous Image Dehazing, Yu Guo, Yuan Gao, Ryan Wen Liu, Yuxu Lu, Jingxiang Qu, Shengfeng He, Ren Wenqi

Research Collection School Of Computing and Information Systems

The presence of non-homogeneous haze can cause scene blurring, color distortion, low contrast, and other degradations that obscure texture details. Existing homogeneous dehazing methods struggle to handle the non-uniform distribution of haze in a robust manner. The crucial challenge of non-homogeneous dehazing is to effectively extract the non-uniform distribution features and reconstruct the details of hazy areas with high quality. In this paper, we propose a novel self-paced semi-curricular attention network, called SCANet, for non-homogeneous image dehazing that focuses on enhancing haze-occluded regions. Our approach consists of an attention generator network and a scene re-construction network. We use the luminance …

Go to article

Mosaic: Spatially-Multiplexed Edge Ai Optimization Over Multiple Concurrent Video Sensing Streams, Ila Gokarn, Hemanth Sabbella, Yigong Hu, Tarek Abdelzaher, Archan Misra Jun 2023

Mosaic: Spatially-Multiplexed Edge Ai Optimization Over Multiple Concurrent Video Sensing Streams, Ila Gokarn, Hemanth Sabbella, Yigong Hu, Tarek Abdelzaher, Archan Misra

Research Collection School Of Computing and Information Systems

Sustaining high fidelity and high throughput of perception tasks over vision sensor streams on edge devices remains a formidable challenge, especially given the continuing increase in image sizes (e.g., generated by 4K cameras) and complexity of DNN models. One promising approach involves criticality-aware processing, where the computation is directed selectively to "critical" portions of individual image frames. We introduce MOSAIC, a novel system for such criticality-aware concurrent processing of multiple vision sensing streams that provides a multiplicative increase in the achievable throughput with negligible loss in perception fidelity. MOSAIC determines critical regions from images received from multiple vision …

Go to article

Position-Guided Text Prompt For Vision-Language Pre-Training, Alex Jinpeng Wang, Pan Zhou, Mike Zheng Shou, Yan Shuicheng Jun 2023

Position-Guided Text Prompt For Vision-Language Pre-Training, Alex Jinpeng Wang, Pan Zhou, Mike Zheng Shou, Yan Shuicheng

Research Collection School Of Computing and Information Systems

Vision-Language Pre-Training (VLP) has shown promising capabilities to align image and text pairs, facilitating a broad variety of cross-modal learning tasks. However, we observe that VLP models often lack the visual grounding/localization capability which is critical for many downstream tasks such as visual reasoning. In this work, we propose a novel Position-guided Text Prompt (PTP) paradigm to enhance the visual grounding ability of cross-modal models trained with VLP. Specifically, in the VLP phase, PTP divides the image into N x N blocks, and identifies the objects in each block through the widely used object detector in VLP. It then reformulates …

Go to article

Curricular Contrastive Regularization For Physics-Aware Single Image Dehazing, Yu Zheng, Jiahui Zhan, Shengfeng He, Yong Du Jun 2023

Curricular Contrastive Regularization For Physics-Aware Single Image Dehazing, Yu Zheng, Jiahui Zhan, Shengfeng He, Yong Du

Research Collection School Of Computing and Information Systems

Considering the ill-posed nature, contrastive regularization has been developed for single image dehazing, introducing the information from negative images as a lower bound. However, the contrastive samples are non-consensual, as the negatives are usually represented distantly from the clear (i.e., positive) image, leaving the solution space still under-constricted. Moreover, the interpretability of deep dehazing models is underexplored towards the physics of the hazing process. In this paper, we propose a novel curricular contrastive regularization targeted at a consensual contrastive space as opposed to a non-consensual one. Our negatives, which provide better lower-bound constraints, can be assembled from 1) the hazy …

Go to article

First Prev Next Last