Interstice,
2023
Rhode Island School of Design
Interstice, Shravan Rao
Masters Theses
When I was about three years old, I distinctly remember being too small to see what was on top of the table. A couple of years later, when I could see those objects, I thought the world around me had grown smaller. In a way, it did, as I experienced, lived, captured, remembered, and shared the space repeatedly. This sense of the world shrinking was exaggerated during the Covid-19 pandemic, allowing new behaviours and modes of interaction to emerge. Continually shaping our modern lives, virtual technologies redefine how we access and share information and stories or even explore new places. …
An Investigation Into Machine Learning Techniques For Designing Dynamic Difficulty Agents In Real-Time Games,
2023
University of Denver
An Investigation Into Machine Learning Techniques For Designing Dynamic Difficulty Agents In Real-Time Games, Ryan Adare Dunagan
Electronic Theses and Dissertations
Video games are an incredibly popular pastime enjoyed by people of all ages world wide. Many different kinds of games exist, but most games feature some elements of the player overcoming some challenge, usually through gameplay. These challenges are insurmountable for some people and may turn them off to video games as a pastime. Games can be made more accessible to players of little skill and/or experience through the use of Dynamic Difficulty Adjustment (DDA) systems that adjust the difficulty of the game in response to the player’s performance. This research seeks to establish the effectiveness of machine learning techniques …
Generalizing Graph Neural Networks Across Graphs, Time, And Tasks,
2023
Singapore Management University
Generalizing Graph Neural Networks Across Graphs, Time, And Tasks, Zhihao Wen
Dissertations and Theses Collection (Open Access)
Graph-structured data are ubiquitous across numerous real-world contexts, encompassing social networks, commercial graphs, bibliographic networks, and biological systems. Delving into the analysis of these graphs can yield significant understanding pertaining to their corresponding application fields.Graph representation learning offers a potent solution to graph analytics challenges by transforming a graph into a low-dimensional space while preserving its information to the greatest extent possible. This conversion into low-dimensional vectors enables the efficient computation of subsequent graph algorithms. The majority of prior research has concentrated on deriving node representations from a single, static graph. However, numerous real-world situations demand rapid generation of representations …
Position-Guided Text Prompt For Vision-Language Pre-Training,
2023
Singapore Management University
Position-Guided Text Prompt For Vision-Language Pre-Training, Alex Jinpeng Wang, Pan Zhou, Mike Zheng Shou, Yan Shuicheng
Research Collection School Of Computing and Information Systems
Vision-Language Pre-Training (VLP) has shown promising capabilities to align image and text pairs, facilitating a broad variety of cross-modal learning tasks. However, we observe that VLP models often lack the visual grounding/localization capability which is critical for many downstream tasks such as visual reasoning. In this work, we propose a novel Position-guided Text Prompt (PTP) paradigm to enhance the visual grounding ability of cross-modal models trained with VLP. Specifically, in the VLP phase, PTP divides the image into N x N blocks, and identifies the objects in each block through the widely used object detector in VLP. It then reformulates …
Class-Incremental Exemplar Compression For Class-Incremental Learning,
2023
Singapore Management University
Class-Incremental Exemplar Compression For Class-Incremental Learning, Zilin Luo, Yaoyao Liu, Bernt Schiele, Qianru Sun
Research Collection School Of Computing and Information Systems
Exemplar-based class-incremental learning (CIL) finetunes the model with all samples of new classes but few-shot exemplars of old classes in each incremental phase, where the "few-shot" abides by the limited memory budget. In this paper, we break this "few-shot" limit based on a simple yet surprisingly effective idea: compressing exemplars by downsampling non-discriminative pixels and saving "many-shot" compressed exemplars in the memory. Without needing any manual annotation, we achieve this compression by generating 0-1 masks on discriminative pixels from class activation maps (CAM). We propose an adaptive mask generation model called class-incremental masking (CIM) to explicitly resolve two difficulties of …
Extracting Class Activation Maps From Non-Discriminative Features As Well,
2023
Singapore Management University
Extracting Class Activation Maps From Non-Discriminative Features As Well, Zhaozheng Chen, Qianru Sun
Research Collection School Of Computing and Information Systems
Extracting class activation maps (CAM) from a classification model often results in poor coverage on foreground objects, i.e., only the discriminative region (e.g., the “head” of “sheep”) is recognized and the rest (e.g., the “leg” of “sheep”) mistakenly as background. The crux behind is that the weight of the classifier (used to compute CAM) captures only the discriminative features of objects. We tackle this by introducing a new computation method for CAM that explicitly captures non-discriminative features as well, thereby expanding CAM to cover whole objects. Specifically, we omit the last pooling layer of the classification model, and perform clustering …
Freestyle Layout-To-Image Synthesis,
2023
Singapore Management University
Freestyle Layout-To-Image Synthesis, Han Xue, Zhiwu Huang, Qianru Sun, Li Song, Wenjun Zhang
Research Collection School Of Computing and Information Systems
Typical layout-to-image synthesis (LIS) models generate images for a close set of semantic classes, e.g., 182 common objects in COCO-Stuff. In this work, we explore the freestyle capability of the model, i.e., how far can it generate unseen semantics (e.g., classes, attributes, and styles) onto a given layout, and call the task Freestyle LIS (FLIS). Thanks to the development of large-scale pre-trained language-image models, a number of discriminative models (e.g., image classification and object detection) trained on limited base classes are empowered with the ability of unseen class prediction. Inspired by this, we opt to leverage large-scale pre-trained text-to-image diffusion …
Scanet: Self-Paced Semi-Curricular Attention Network For Non-Homogeneous Image Dehazing,
2023
Singapore Management University
Scanet: Self-Paced Semi-Curricular Attention Network For Non-Homogeneous Image Dehazing, Yu Guo, Yuan Gao, Ryan Wen Liu, Yuxu Lu, Jingxiang Qu, Shengfeng He, Ren Wenqi
Research Collection School Of Computing and Information Systems
The presence of non-homogeneous haze can cause scene blurring, color distortion, low contrast, and other degradations that obscure texture details. Existing homogeneous dehazing methods struggle to handle the non-uniform distribution of haze in a robust manner. The crucial challenge of non-homogeneous dehazing is to effectively extract the non-uniform distribution features and reconstruct the details of hazy areas with high quality. In this paper, we propose a novel self-paced semi-curricular attention network, called SCANet, for non-homogeneous image dehazing that focuses on enhancing haze-occluded regions. Our approach consists of an attention generator network and a scene re-construction network. We use the luminance …
Semantic Scene Completion With Cleaner Self,
2023
Singapore Management University
Semantic Scene Completion With Cleaner Self, Fengyun Wang, Dong Zhang, Hanwang Zhang, Jinhui Tang, Qianru Sun
Research Collection School Of Computing and Information Systems
Semantic Scene Completion (SSC) transforms an image of single-view depth and/or RGB 2D pixels into 3D voxels, each of whose semantic labels are predicted. SSC is a well-known ill-posed problem as the prediction model has to “imagine” what is behind the visible surface, which is usually represented by Truncated Signed Distance Function (TSDF). Due to the sensory imperfection of the depth camera, most existing methods based on the noisy TSDF estimated from depth values suffer from 1) incomplete volumetric predictions and 2) confused semantic labels. To this end, we use the ground-truth 3D voxels to generate a perfect visible surface, …
Unbiased Multiple Instance Learning For Weakly Supervised Video Anomaly Detection,
2023
Singapore Management University
Unbiased Multiple Instance Learning For Weakly Supervised Video Anomaly Detection, Hui Lyu, Zhongqi Yue, Qianru Sun, Bin Luo, Zhen Cui, Hanwang Zhang
Research Collection School Of Computing and Information Systems
Weakly Supervised Video Anomaly Detection (WSVAD) is challenging because the binary anomaly label is only given on the video level, but the output requires snippet-level predictions. So, Multiple Instance Learning (MIL) is prevailing in WSVAD. However, MIL is notoriously known to suffer from many false alarms because the snippet-level detector is easily biased towards the abnormal snippets with simple context, confused by the normality with the same bias, and missing the anomaly with a different pattern. To this end, we propose a new MIL framework: Unbiased MIL (UMIL), to learn unbiased anomaly features that improve WSVAD. At each MIL training …
Evading Deepfake Detectors Via Adversarial Statistical Consistency,
2023
Singapore Management University
Evading Deepfake Detectors Via Adversarial Statistical Consistency, Yang Hou, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, Jianjun Zhao
Research Collection School Of Computing and Information Systems
In recent years, as various realistic face forgery techniques known as DeepFake improves by leaps and bounds, more and more DeepFake detection techniques have been proposed. These methods typically rely on detecting statistical differences between natural (i.e., real) and DeepFake-generated images in both spatial and frequency domains. In this work, we propose to explicitly minimize the statistical differences to evade state-of-the-art DeepFake detectors. To this end, we propose a statistical consistency attack (StatAttack) against DeepFake detectors, which contains two main parts. First, we select several statistical-sensitive natural degradations (i.e., exposure, blur, and noise) and add them to the fake images …
Curricular Contrastive Regularization For Physics-Aware Single Image Dehazing,
2023
Singapore Management University
Curricular Contrastive Regularization For Physics-Aware Single Image Dehazing, Yu Zheng, Jiahui Zhan, Shengfeng He, Yong Du
Research Collection School Of Computing and Information Systems
Considering the ill-posed nature, contrastive regularization has been developed for single image dehazing, introducing the information from negative images as a lower bound. However, the contrastive samples are non-consensual, as the negatives are usually represented distantly from the clear (i.e., positive) image, leaving the solution space still under-constricted. Moreover, the interpretability of deep dehazing models is underexplored towards the physics of the hazing process. In this paper, we propose a novel curricular contrastive regularization targeted at a consensual contrastive space as opposed to a non-consensual one. Our negatives, which provide better lower-bound constraints, can be assembled from 1) the hazy …
Towards A Smaller Student: Capacity Dynamic Distillation For Efficient Image Retrieval,
2023
Singapore Management University
Towards A Smaller Student: Capacity Dynamic Distillation For Efficient Image Retrieval, Yi Xie, Huaidong Zhang, Xuemiao Xu, Jianqing Zhu, Shengfeng He
Research Collection School Of Computing and Information Systems
Previous Knowledge Distillation based efficient image retrieval methods employ a lightweight network as the student model for fast inference. However, the lightweight student model lacks adequate representation capacity for effective knowledge imitation during the most critical early training period, causing final performance degeneration. To tackle this issue, we propose a Capacity Dynamic Distillation framework, which constructs a student model with editable representation capacity. Specifically, the employed student model is initially a heavy model to fruitfully learn distilled knowledge in the early training epochs, and the student model is gradually compressed during the training. To dynamically adjust the model capacity, our …
Venus: A Geometrical Representation For Quantum State Visualization,
2023
Singapore Management University
Venus: A Geometrical Representation For Quantum State Visualization, Shaolun Ruan, Ribo Yuan, Qiang Guan, Yanna Lin, Ying Mao, Weiwen Jiang, Zhepeng Wang, Wei Xu, Yong Wang
Research Collection School Of Computing and Information Systems
Visualizations have played a crucial role in helping quantum computing users explore quantum states in various quantum computing applications. Among them, Bloch Sphere is the widely-used visualization for showing quantum states, which leverages angles to represent quantum amplitudes. However, it cannot support the visualization of quantum entanglement and superposition, the two essential properties of quantum computing. To address this issue, we propose VENUS, a novel visualization for quantum state representation. By explicitly correlating 2D geometric shapes based on the math foundation of quantum computing characteristics, VENUS effectively represents quantum amplitudes of both the single qubit and two qubits for quantum …
Ifundit: Visual Profiling Of Fund Investment Styles,
2023
Singapore Management University
Ifundit: Visual Profiling Of Fund Investment Styles, Rong Zhang, Bon Kyung Ku, Yong Wang, Xuanwu Yue, Siyuan Liu, Ke Li, Huamin Qu
Research Collection School Of Computing and Information Systems
Mutual funds are becoming increasingly popular with the emergence of Internet finance. Clear profiling of a fund's investment style is crucial for fund managers to evaluate their investment strategies, and for investors to understand their investment. However, it is challenging to profile a fund's investment style as it requires a comprehensive analysis of complex multi-dimensional temporal data. In addition, different fund managers and investors have different focuses when analysing a fund's investment style. To address the issue, we propose iFUNDit, an interactive visual analytic system for fund investment style analysis. The system decomposes a fund's critical features into performance attributes …
Gnnlens: A Visual Analytics Approach For Prediction Error Diagnosis Of Graph Neural Networks.,
2023
Hong Kong University of Science and Technology
Gnnlens: A Visual Analytics Approach For Prediction Error Diagnosis Of Graph Neural Networks., Zhihua Jin, Yong Wang, Qianwen Wang, Yao Ming, Tengfei Ma, Huamin Qu
Research Collection School Of Computing and Information Systems
Graph Neural Networks (GNNs) aim to extend deep learning techniques to graph data and have achieved significant progress in graph analysis tasks (e.g., node classification) in recent years. However, similar to other deep neural networks like Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), GNNs behave like a black box with their details hidden from model developers and users. It is therefore difficult to diagnose possible errors of GNNs. Despite many visual analytics studies being done on CNNs and RNNs, little research has addressed the challenges for GNNs. This paper fills the research gap with an interactive visual analysis …
Mosaic: Spatially-Multiplexed Edge Ai Optimization Over Multiple Concurrent Video Sensing Streams,
2023
Singapore Management University
Mosaic: Spatially-Multiplexed Edge Ai Optimization Over Multiple Concurrent Video Sensing Streams, Ila Gokarn, Hemanth Sabbella, Yigong Hu, Tarek Abdelzaher, Archan Misra
Research Collection School Of Computing and Information Systems
Sustaining high fidelity and high throughput of perception tasks over vision sensor streams on edge devices remains a formidable challenge, especially given the continuing increase in image sizes (e.g., generated by 4K cameras) and complexity of DNN models. One promising approach involves criticality-aware processing, where the computation is directed selectively to "critical" portions of individual image frames. We introduce MOSAIC, a novel system for such criticality-aware concurrent processing of multiple vision sensing streams that provides a multiplicative increase in the achievable throughput with negligible loss in perception fidelity. MOSAIC determines critical regions from images received from multiple vision …
Human And Technical Factors In The Adoption Of Quantum Cryptographic Algorithms,
2023
Rose-Hulman Institute of Technology
Human And Technical Factors In The Adoption Of Quantum Cryptographic Algorithms, Alyssa Pinkston
Mathematical Sciences Technical Reports (MSTR)
The purpose of this research is to understand what factors would cause users to choose quantum key distribution (QKD) over other methods of cryptography. An Advanced Encryption Standard (AES) key can be exchanged through communication using the Rivest, Shamir, Adleman (RSA) cryptographic algorithm, QKD, or post-quantum cryptography (PQC). QKD relies on quantum physics where RSA and PQC use complex mathematics to encrypt data. The BB84 quantum cryptographic protocol involves communication over a quantum channel and a public channel. The quantum channel can be technically attacked by beamsplitting or intercept/resend. QKD, like other forms of cryptography, is vulnerable to social attacks …
An Assistive Interface For Displaying Novice's Code History,
2023
Washington University in St. Louis
An Assistive Interface For Displaying Novice's Code History, Ruiwei Xiao
McKelvey School of Engineering Graduate Student Theses & Dissertations
As Teaching Assistant (TA) programs grow in number and size in introductory CS courses, TAs play a significant role in novice programmers' experience and contribute to their success. However, many TAs are also relative beginners themselves and thus have limited experience in programming and teaching. Thus the effectiveness and consistency of their guidance can vary significantly. To improve interaction quality and assist TAs in providing better support, we examine the difficulties encountered by inexperienced TAs in previous literature and then identify the potential for the high cognitive load as an unaddressed difficulty that may prevent new TAs from initiating effective …
Expressive Marks: Art In The Age Of Augmented Reality,
2023
Dartmouth College
Expressive Marks: Art In The Age Of Augmented Reality, Carson G. Levine
Dartmouth College Master’s Theses
Augmented reality (AR) and non-fungible tokens (NFTs) introduce new considerations for the long-standing debate of what it means for digital art to be “real.” However, the ability to create AR experiences is limited to those who are technically skilled or who can afford to consult someone else. This paper addresses the need for an accessible tool that enables artists of all technical backgrounds to expressively create marks in AR. The solution includes a mobile application called CrayonAR. The system was designed to be modular, minimal, and physically engaging, and was developed in Unity using ARFoundation and Firebase Storage and Realtime …
