Faithful Extreme Rescaling Via Generative Prior Reciprocated Invertible Representations,
2022
Singapore Management University
Faithful Extreme Rescaling Via Generative Prior Reciprocated Invertible Representations, Zhixuan Zhong, Liangyu Chai, Yang Zhou, Bailin Deng, Jia Pan, Shengfeng He
Research Collection School Of Computing and Information Systems
This paper presents a Generative prior ReciprocAted Invertible rescaling Network (GRAIN) for generating faithful high-resolution (HR) images from low-resolution (LR) invertible images with an extreme upscaling factor (64×). Previous researches have leveraged the prior knowledge of a pretrained GAN model to generate high-quality upscaling results. However, they fail to produce pixel-accurate results due to the highly ambiguous extreme mapping process. We remedy this problem by introducing a reciprocated invertible image rescaling process, in which high-resolution information can be delicately embedded into an invertible low-resolution image and generative prior for a faithful HR reconstruction. In particular, the invertible LR features not …
Group Contextualization For Video Recognition,
2022
University of Science and Technology of China
Group Contextualization For Video Recognition, Yanbin Hao, Hao Zhang, Chong-Wah Ngo, Xiangnan He
Research Collection School Of Computing and Information Systems
Learning discriminative representation from the complex spatio-temporal dynamic space is essential for video recognition. On top of those stylized spatio-temporal computational units, further refining the learnt feature with axial contexts is demonstrated to be promising in achieving this goal. However, previous works generally focus on utilizing a single kind of contexts to calibrate entire feature channels and could hardly apply to deal with diverse video activities. The problem can be tackled by using pair-wise spatio-temporal attentions to recompute feature response with cross-axis contexts at the expense of heavy computations. In this paper, we propose an efficient feature refinement method that …
Revisiting Local Descriptor For Improved Few-Shot Classification,
2022
Singapore Management University
Revisiting Local Descriptor For Improved Few-Shot Classification, Jun He, Richang Hong, Xueliang Liu, Mingliang Xu, Qianru Sun
Research Collection School Of Computing and Information Systems
Few-shot classification studies the problem of quickly adapting a deep learner to understanding novel classes based on few support images. In this context, recent research efforts have been aimed at designing more and more complex classifiers that measure similarities between query and support images but left the importance of feature embeddings seldom explored. We show that the reliance on sophisticated classifiers is not necessary, and a simple classifier applied directly to improved feature embeddings can instead outperform most of the leading methods in the literature. To this end, we present a new method, named DCAP, for few-shot classification, in which …
Flavor-Videos: Enhancing The Flavor Perception Of Food While Eating With Videos,
2022
Singapore Management University
Flavor-Videos: Enhancing The Flavor Perception Of Food While Eating With Videos, Meetha Nesam James, Nimesha Ranasinghe, Anthony Tang, Lora Oehlberg
Research Collection School Of Computing and Information Systems
People are typically involved in different activities while eating, particularly when eating alone, such as watching television or playing games on their phones. Previous research in Human-Food Interaction (HFI) has primarily focused on studying people’s motivation and analyzing of the media content watched while eating. However, their impact on human behavioral and cognitive processes, particularly flavor perception and its attributes, remains underexplored. We present a user study to investigate the influence of six types of videos, including mukbang – a new food video genre, on flavor perceptions (taste sensations, liking, and emotions) while eating plain white rice. Our findings revealed …
Out-Of-Core Gpu Path Tracing On Large Instanced Scenes Via Geometry Streaming,
2022
California Polytechnic State University, San Luis Obispo
Out-Of-Core Gpu Path Tracing On Large Instanced Scenes Via Geometry Streaming, Jeremy Berchtold
Master's Theses
We present a technique for out-of-core GPU path tracing of arbitrarily large scenes that is compatible with hardware-accelerated ray-tracing. Our technique improves upon previous works by subdividing the scene spatially into streamable chunks that are loaded using a priority system that maximizes ray throughput and minimizes GPU memory usage. This allows for arbitrarily large scaling of scene complexity. Our system required under 19 minutes to render a solid color version of Disney's Moana Island scene (39.3 million instances, 261.1 million unique quads, and 82.4 billion instanced quads at a resolution of 1024x429 and 1024spp on an RTX 5000 (24GB memory …
Mlp-3d: A Mlp-Like 3d Architecture With Grouped Time Mixing,
2022
Singapore Management University
Mlp-3d: A Mlp-Like 3d Architecture With Grouped Time Mixing, Zhaofan Qiu, Ting Yao, Chong-Wah Ngo, Tao Mei
Research Collection School Of Computing and Information Systems
Convolutional Neural Networks (CNNs) have been re-garded as the go-to models for visual recognition. More re-cently, convolution-free networks, based on multi-head self-attention (MSA) or multi-layer perceptrons (MLPs), become more and more popular. Nevertheless, it is not trivial when utilizing these newly-minted networks for video recognition due to the large variations and complexities in video data. In this paper, we present MLP-3D networks, a novel MLP-like 3D architecture for video recognition. Specifically, the architecture consists of MLP-3D blocks, where each block contains one MLP applied across tokens (i.e., token-mixing MLP) and one MLP applied independently to each token (i.e., channel MLP). …
A Bidirectional Formulation For Walk On Spheres,
2022
Dartmouth College
A Bidirectional Formulation For Walk On Spheres, Yang Qi
Dartmouth College Master’s Theses
Poisson’s equations and Laplace’s equations are important linear partial differential equations (PDEs)
widely used in many applications. Conventional methods for solving PDEs numerically often need to
discretize the space first, making them less efficient for complex shapes. The random walk on spheres
method (WoS) is a grid-free Monte-Carlo method for solving PDEs that does not need to discrete the
space. We draw analogies between WoS and classical rendering algorithms, and find that the WoS
algorithm is conceptually identical to forward path tracing.
We show that solving the Poisson’s equation is equivalent to solving the Green’s function for every
pair of …
Adaptive Temporal Grouping For Black-Box Adversarial Attacks On Videos,
2022
Singapore Management University
Adaptive Temporal Grouping For Black-Box Adversarial Attacks On Videos, Zhipeng Wei, Jingjing Chen, Hao Zhang, Linxi Jiang, Yu-Gang Jiang
Research Collection School Of Computing and Information Systems
Deep-learning based video models, which have remarkable performance on action recognition tasks, are recently proved to be vulnerable to adversarial samples, even those generated in the black-box setting. However, these black-box attack methods are insufficient to attack videos models in real-world applications due to the requirement of lots of queries. To this end, we propose to boost the efficiency of black-box attacks on video recognition models. Although videos carry rich temporal information, they include redundant spatial information from adjacent frames. This motivates us to introduce the adaptive temporal grouping (ATG) method, which groups video frames by the similarity of their …
Multimodal Zero-Shot Hateful Meme Detection,
2022
Singapore Management University
Multimodal Zero-Shot Hateful Meme Detection, Jiawen Zhu, Roy Ka-Wei Lee, Wen Haw Chong
Research Collection School Of Computing and Information Systems
Facebook has recently launched the hateful meme detection challenge, which garnered much attention in academic and industry research communities. Researchers have proposed multimodal deep learning classification methods to perform hateful meme detection. While the proposed methods have yielded promising results, these classification methods are mostly supervised and heavily rely on labeled data that are not always available in the real-world setting. Therefore, this paper explores and aims to perform hateful meme detection in a zero-shot setting. Working towards this goal, we propose Target-Aware Multimodal Enhancement (TAME), which is a novel deep generative framework that can improve existing hateful meme classification …
A Simple Data Mixing Prior For Improving Self-Supervised Learning,
2022
Singapore Management University
A Simple Data Mixing Prior For Improving Self-Supervised Learning, Sucheng Ren, Huiyu Wang, Zhengqi Gao, Shengfeng He, Alan Yuille, Yuyin Zhou, Cihang Xie
Research Collection School Of Computing and Information Systems
Data mixing (e.g., Mixup, Cutmix, ResizeMix) is an essential component for advancing recognition models. In this paper, we focus on studying its effectiveness in the self-supervised setting. By noticing the mixed images that share the same source images are intrinsically related to each other, we hereby propose SDMP, short for Simple Data Mixing Prior, to capture this straightforward yet essential prior, and position such mixed images as additional positive pairs to facilitate self-supervised representation learning. Our experiments verify that the proposed SDMP enables data mixing to help a set of self-supervised learning frameworks (e.g., MoCo) achieve better accuracy and out-of-distribution …
High-Resolution Face Swapping Via Latent Semantics Disentanglement,
2022
Singapore Management University
High-Resolution Face Swapping Via Latent Semantics Disentanglement, Yangyang Xu, Bailin Deng, Junle Wang, Yanqing Jing, Jia Pan, Shengfeng He
Research Collection School Of Computing and Information Systems
We present a novel high-resolution face swapping method using the inherent prior knowledge of a pre-trained GAN model. Although previous research can leverage generative priors to produce high-resolution results, their quality can suffer from the entangled semantics of the latent space. We explicitly disentangle the latent semantics by utilizing the progressive nature of the generator, deriving structure at-tributes from the shallow layers and appearance attributes from the deeper ones. Identity and pose information within the structure attributes are further separated by introducing a landmark-driven structure transfer latent direction. The disentangled latent code produces rich generative features that incorporate feature blending …
Cross-Lingual Adaptation For Recipe Retrieval With Mixup,
2022
Singapore Management University
Cross-Lingual Adaptation For Recipe Retrieval With Mixup, Bin Zhu, Chong-Wah Ngo, Jingjing Chen, Wing-Kwong Chan
Research Collection School Of Computing and Information Systems
Cross-modal recipe retrieval has attracted research attention in recent years, thanks to the availability of large-scale paired data for training. Nevertheless, obtaining adequate recipe-image pairs covering the majority of cuisines for supervised learning is difficult if not impossible. By transferring knowledge learnt from a data-rich cuisine to a data-scarce cuisine, domain adaptation sheds light on this practical problem. Nevertheless, existing works assume recipes in source and target domains are mostly originated from the same cuisine and written in the same language. This paper studies unsupervised domain adaptation for image-to-recipe retrieval, where recipes in source and target domains are in different …
Reinforcement Learning-Based Interactive Video Search,
2022
Singapore Management University
Reinforcement Learning-Based Interactive Video Search, Zhixin Ma, Jiaxin Wu, Zhijian Hou, Chong-Wah Ngo
Research Collection School Of Computing and Information Systems
Despite the rapid progress in text-to-video search due to the advancement of cross-modal representation learning, the existing techniques still fall short in helping users to rapidly identify the search targets. Particularly, in the situation that a system suggests a long list of similar candidates, the user needs to painstakingly inspect every search result. The experience is frustrated with repeated watching of similar clips, and more frustratingly, the search targets may be overlooked due to mental tiredness. This paper explores reinforcement learning-based (RL) searching to relieve the user from the burden of brute force inspection. Specifically, the system maintains a graph …
Increasing Perceived Realism Of Objects In A Mixed Reality Environment Using 'Diminished Virtual Reality',
2022
University of Mississippi
Increasing Perceived Realism Of Objects In A Mixed Reality Environment Using 'Diminished Virtual Reality', Logan Scott Parker
Honors Theses
With the recent explosion of popularity of virtual and mixed reality, an important question has arisen: “Is there a way to create a better blend of real and virtual worlds in a mixed reality experience?” This research attempts to determine whether a visual filter can be created and applied to virtual objects to better convince the brain into interpreting a composite of virtual and real views as one seamless view. The method devised in this thesis is being called 'Diminished Virtual Reality'. The results found in this study show that when presented with a scene composed of a combination of …
Pluto In Hand: Design And Implementation Of A Location-Based Mobile Augmented Reality Application For Viewing Open Data,
2022
CUNY Hunter College
Pluto In Hand: Design And Implementation Of A Location-Based Mobile Augmented Reality Application For Viewing Open Data, Matthew O. Ward
Theses and Dissertations
Immersive mobile augmented reality (AR) technology has improved while geolocational data volume has grown. City governments can utilize this technology to share their geospatial data with the public, promoting smart city aims. This research describes the design and implementation of a novel open-source ARGIS application to view property tax lot information in New York City. This proof-of-technology demonstrates web-based AR can visualize location-based spatial data.
Automated Filament Inking For Multi-Color Fff 3d Printing,
2022
Dartmouth College
Automated Filament Inking For Multi-Color Fff 3d Printing, Eammon Littler
Dartmouth College Master’s Theses
We propose a novel system for low-cost multi-color Fused Filament Fabrication (FFF) 3D printing, allowing for the creation of customizable colored filament using a pre-processing approach. We developed an open-source device to automatically ink filament using permanent markers. Our device can be built using 3D printed parts and off-the-shelf electronics. An accompanying web-based interface allows users to view GCODE toolpaths for a multi-color print and quickly generate filament color profiles. Taking a pre-processing approach makes this system compatible with the majority of desktop 3D printers on the market, as the processed filament behaves no differently from conventional filaments. Furthermore, inked …
Gauging The State-Of-The-Art For Foresight Weight Pruning On Neural Networks,
2022
University of Arkansas, Fayetteville
Gauging The State-Of-The-Art For Foresight Weight Pruning On Neural Networks, Noah James
Computer Science and Computer Engineering Undergraduate Honors Theses
The state-of-the-art for pruning neural networks is ambiguous due to poor experimental practices in the field. Newly developed approaches rarely compare to each other, and when they do, their comparisons are lackluster or contain errors. In the interest of stabilizing the field of pruning, this paper initiates a dive into reproducing prominent pruning algorithms across several architectures and datasets. As a first step towards this goal, this paper shows results for foresight weight pruning across 6 baseline pruning strategies, 5 modern pruning strategies, random pruning, and one legacy method (Optimal Brain Damage). All strategies are evaluated on 3 different architectures …
An Investigation Into, And The Construction Of, An Operable Windows Notifier,
2022
University of Arkansas, Fayetteville
An Investigation Into, And The Construction Of, An Operable Windows Notifier, Grey Hixson
Computer Science and Computer Engineering Undergraduate Honors Theses
The Office of Sustainability at the University of Arkansas identified that building occupants that have control over operable windows may open them at inappropriate times. Windows opened in a building with a temperature and air differential leads to increased HVAC operating costs and building occupant discomfort. This led the Associate Vice Chancellor of Facilities at the University of Arkansas to propose the construction of a mobile application that a building occupant can use to make an informed decision before opening their window. I have formulated a series of research objectives in conjunction with the Director of the Office of Sustainability …
Analysis Of Gpu Memory Vulnerabilities,
2022
University of Arkansas, Fayetteville
Analysis Of Gpu Memory Vulnerabilities, Jarrett Hoover
Computer Science and Computer Engineering Undergraduate Honors Theses
Graphics processing units (GPUs) have become a widely used technology for various purposes. While their intended use is accelerating graphics rendering, their parallel computing capabilities have expanded their use into other areas. They are used in computer gaming, deep learning for artificial intelligence and mining cryptocurrencies. Their rise in popularity led to research involving several security aspects, including this paper’s focus, memory vulnerabilities. Research documented many vulnerabilities, including GPUs not implementing address space layout randomization, not zeroing out memory after deallocation, and not initializing newly allocated memory. These vulnerabilities can lead to a victim’s sensitive data being leaked to an …
Using A Bert-Based Ensemble Network For Abusive Language Detection,
2022
University of Arkansas, Fayetteville
Using A Bert-Based Ensemble Network For Abusive Language Detection, Noah Ballinger
Computer Science and Computer Engineering Undergraduate Honors Theses
Over the past two decades, online discussion has skyrocketed in scope and scale. However, so has the amount of toxicity and offensive posts on social media and other discussion sites. Despite this rise in prevalence, the ability to automatically moderate online discussion platforms has seen minimal development. Recently, though, as the capabilities of artificial intelligence (AI) continue to improve, the potential of AI-based detection of harmful internet content has become a real possibility. In the past couple years, there has been a surge in performance on tasks in the field of natural language processing, mainly due to the development of …
