Computer Sciences Commons^™

Codem: Conditional Domain Embeddings For Scalable Human Activity Recognition, Abu Zaher Md Faridee, Avijoy Chakma, Zahid Hasan, Nirmalya Roy, Archan Misra

Research Collection School Of Computing and Information Systems

We explore the effect of auxiliary labels in improving the classification accuracy of wearable sensor-based human activity recognition (HAR) systems, which are primarily trained with the supervision of the activity labels (e.g. running, walking, jumping). Supplemental meta-data are often available during the data collection process such as body positions of the wearable sensors, subjects' demographic information (e.g. gender, age), and the type of wearable used (e.g. smartphone, smart-watch). This information, while not directly related to the activity classification task, can nonetheless provide auxiliary supervision and has the potential to significantly improve the HAR accuracy by providing extra guidance on how …

Rhythmedge: Enabling Contactless Heart Rate Estimation On The Edge, Zahid Hasan, Emon Dey, Sreenivasan Ramasamy Ramamurthy, Nirmalya Roy, Archan Misra

Research Collection School Of Computing and Information Systems

The primary contribution of this paper is designing and prototyping a real-time edge computing system, RhythmEdge, that is capable of detecting changes in blood volume from facial videos (Remote Photoplethysmography; rPPG), enabling cardio-vascular health assessment instantly. The benefits of RhythmEdge include non-invasive measurement of cardiovascular activity, real-time system operation, inexpensive sensing components, and computing. RhythmEdge captures a short video of the skin using a camera and extracts rPPG features to estimate the Photoplethysmography (PPG) signal using a multi-task learning framework while offloading the edge computation. In addition, we intelligently apply a transfer learning approach to the multi-task learning framework to …

Flavor-Videos: Enhancing The Flavor Perception Of Food While Eating With Videos, Meetha Nesam James, Nimesha Ranasinghe, Anthony Tang, Lora Oehlberg

Research Collection School Of Computing and Information Systems

People are typically involved in different activities while eating, particularly when eating alone, such as watching television or playing games on their phones. Previous research in Human-Food Interaction (HFI) has primarily focused on studying people’s motivation and analyzing of the media content watched while eating. However, their impact on human behavioral and cognitive processes, particularly flavor perception and its attributes, remains underexplored. We present a user study to investigate the influence of six types of videos, including mukbang – a new food video genre, on flavor perceptions (taste sensations, liking, and emotions) while eating plain white rice. Our findings revealed …

Learning To Solve Routing Problems Via Distributionally Robust Optimization, Jiang Yuan, Yaoxin Wu, Zhiguang Cao

Research Collection School Of Computing and Information Systems

Recent deep models for solving routing problems always assume a single distribution of nodes for training, which severely impairs their cross-distribution generalization ability. In this paper, we exploit group distributionally robust optimization (group DRO) to tackle this issue, where we jointly optimize the weights for different groups of distributions and the parameters for the deep model in an interleaved manner during training. We also design a module based on convolutional neural network, which allows the deep model to learn more informative latent pattern among the nodes. We evaluate the proposed approach on two types of wellknown deep models including GCN …

Shunted Self-Attention Via Multi-Scale Token Aggregation, Sucheng Ren, Daquan Zhou, Shengfeng He, Jiashi Feng, Xinchao Wang

Research Collection School Of Computing and Information Systems

Recent Vision Transformer (ViT) models have demonstrated encouraging results across various computer vision tasks, thanks to its competence in modeling long-range dependencies of image patches or tokens via self-attention. These models, however, usually designate the similar receptive fields of each token feature within each layer. Such a constraint inevitably limits the ability of each self-attention layer in capturing multi-scale features, thereby leading to performance degradation in handling images with multiple objects of different scales. To address this issue, we propose a novel and generic strategy, termed shunted selfattention (SSA), that allows ViTs to model the attentions at hybrid scales per …

High-Resolution Face Swapping Via Latent Semantics Disentanglement, Yangyang Xu, Bailin Deng, Junle Wang, Yanqing Jing, Jia Pan, Shengfeng He

Research Collection School Of Computing and Information Systems

We present a novel high-resolution face swapping method using the inherent prior knowledge of a pre-trained GAN model. Although previous research can leverage generative priors to produce high-resolution results, their quality can suffer from the entangled semantics of the latent space. We explicitly disentangle the latent semantics by utilizing the progressive nature of the generator, deriving structure at-tributes from the shallow layers and appearance attributes from the deeper ones. Identity and pose information within the structure attributes are further separated by introducing a landmark-driven structure transfer latent direction. The disentangled latent code produces rich generative features that incorporate feature blending …

Co-Advise: Cross Inductive Bias Distillation, Sucheng Ren, Zhengqi Gao, Tiany Hua, Zihui Xue, Yonglong Tian, Shengfeng He, Hang Zhao

Research Collection School Of Computing and Information Systems

The inductive bias of vision transformers is more relaxed that cannot work well with insufficient data. Knowledge distillation is thus introduced to assist the training of transformers. Unlike previous works, where merely heavy convolution-based teachers are provided, in this paper, we delve into the influence of models inductive biases in knowledge distillation (e.g., convolution and involution). Our key observation is that the teacher accuracy is not the dominant reason for the student accuracy, but the teacher inductive bias is more important. We demonstrate that lightweight teachers with different architectural inductive biases can be used to co-advise the student transformer with …

Hu-Fu: Efficient And Secure Spatial Queries Over Data Federation, Yongxin Tong, Xuchen Pan, Yuxiang Zeng, Yexuan Shi, Chunbo Xue, Zimu Zhou, Xiaofei Zhang, Lei Chen, Yi Xu, Ke Xu, Weifeng Lv

Research Collection School Of Computing and Information Systems

Data isolation has become an obstacle to scale up query processing over big data, since sharing raw data among data owners is often prohibitive due to security concerns. A promising solution is to perform secure queries over a federation of multiple data owners leveraging secure multi-party computation (SMC) techniques, as evidenced by recent federation work over relational data. However, existing solutions are highly inefficient on spatial queries due to excessive secure distance operations for query processing and their usage of general-purpose SMC libraries for secure operation implementation. In this paper, we propose Hu-Fu, the first system for efficient and secure …

Desire: An Efficient Dynamic Cluster-Based Forest Indexing For Similarity Search In Multi-Metric Spaces, Yifan Zhu, Lu Chen, Yunjun Gao, Baihua Zheng, Pengfei Wang

Research Collection School Of Computing and Information Systems

Similarity search fnds similar objects for a given query object based on a certain similarity metric. Similarity search in metric spaces has attracted increasing attention, as the metric space can accommodate any type of data and support fexible distance metrics. However, a metric space only models a single data type with a specifc similarity metric. In contrast, a multi-metric space combines multiple metric spaces to simultaneously model a variety of data types and a collection of associated similarity metrics. Thus, a multi-metric space is capable of performing similarity search over any combination of metric spaces. Many studies focus on indexing …

Imagining New Futures Beyond Predictive Systems In Child Welfare: A Qualitative Study With Impacted Stakeholders, Logan Stapleton, Min Hun Lee, Diana Qing, Marya Wright, Alexandra Chouldechova, Ken Holstein, Zhiwei Steven Wu, Haiyi Zhu

Research Collection School Of Computing and Information Systems

Child welfare agencies across the United States are turning to datadriven predictive technologies (commonly called predictive analytics) which use government administrative data to assist workers’ decision-making. While some prior work has explored impacted stakeholders’ concerns with current uses of data-driven predictive risk models (PRMs), less work has asked stakeholders whether such tools ought to be used in the first place. In this work, we conducted a set of seven design workshops with 35 stakeholders who have been impacted by the child welfare system or who work in it to understand their beliefs and concerns around PRMs, and to engage them …

Cross-Lingual Adaptation For Recipe Retrieval With Mixup, Bin Zhu, Chong-Wah Ngo, Jingjing Chen, Wing-Kwong Chan

Research Collection School Of Computing and Information Systems

Cross-modal recipe retrieval has attracted research attention in recent years, thanks to the availability of large-scale paired data for training. Nevertheless, obtaining adequate recipe-image pairs covering the majority of cuisines for supervised learning is difficult if not impossible. By transferring knowledge learnt from a data-rich cuisine to a data-scarce cuisine, domain adaptation sheds light on this practical problem. Nevertheless, existing works assume recipes in source and target domains are mostly originated from the same cuisine and written in the same language. This paper studies unsupervised domain adaptation for image-to-recipe retrieval, where recipes in source and target domains are in different …

Catching Both Gray And Black Swans: Open-Set Supervised Anomaly Detection, Choubo Ding, Guansong Pang, Chunhua Shen

Research Collection School Of Computing and Information Systems

Despite most existing anomaly detection studies assume the availability of normal training samples only, a few labeled anomaly examples are often available in many real-world applications, such as defect samples identified during random quality inspection, lesion images confirmed by radiologists in daily medical screening, etc. These anomaly examples provide valuable knowledge about the application-specific abnormality, enabling significantly improved detection of similar anomalies in some recent models. However, those anomalies seen during training often do not illustrate every possible class of anomaly, rendering these models ineffective in generalizing to unseen anomaly classes. This paper tackles open-set supervised anomaly detection, in which …

Revisiting Local Descriptor For Improved Few-Shot Classification, Jun He, Richang Hong, Xueliang Liu, Mingliang Xu, Qianru Sun

Research Collection School Of Computing and Information Systems

Few-shot classification studies the problem of quickly adapting a deep learner to understanding novel classes based on few support images. In this context, recent research efforts have been aimed at designing more and more complex classifiers that measure similarities between query and support images but left the importance of feature embeddings seldom explored. We show that the reliance on sophisticated classifiers is not necessary, and a simple classifier applied directly to improved feature embeddings can instead outperform most of the leading methods in the literature. To this end, we present a new method, named DCAP, for few-shot classification, in which …

Learnings From A Pilot Hybrid Question Answering System: Cqas: Case Study Based On A Singapore Government Agency's Customer Service Centre, Hui Shan Lee, Shankararaman, Venky, Eng Lieh Ouh

Research Collection School Of Computing and Information Systems

The Singapore Government first released their digital government blueprint in 2018 with the key message for all their agencies to be "digital to the core and served with heart". With this push, agencies are moving towards human-centric digital services, especially for individual citizens. During COVID-19, Singapore government agencies introduced many COVID-19 digital initiatives resulting in more incoming inquiries from citizens to respective agencies. This surge in inquiries created the challenge on the agencies' end to meet service level agreements. One widely adopted solution is the use of chatbot technology that directly interfaces with the customer. However, several organisations have faced …

Joint Pricing And Matching For City-Scale Ride Pooling, Sanket Shah, Meghna Lowalekar, Pradeep Varakantham

Research Collection School Of Computing and Information Systems

Central to efficient ride-pooling are two challenges: (1) how to `price' customers' requests for rides, and (2) if the customer agrees to that price, how to best `match' these requests to drivers. While both of them are interdependent, each challenge's individual complexity has meant that, historically, they have been decoupled and studied individually. This paper creates a framework for batched pricing and matching in which pricing is seen as a meta-level optimisation over different possible matching decisions. Our key contributions are in developing a variant of the revenue-maximizing auction corresponding to the meta-level optimization problem, and then providing a scalable …

Blocklens: Visual Analytics Of Student Coding Behaviors In Block-Based Programming Environments., Sean Tung, Huan Wei, Haotian Li, Yong Wang, Meng Xia, Huamin. Qu

Research Collection School Of Computing and Information Systems

Block-based programming environments have been widely used to introduce K-12 students to coding. To guide students effectively, instructors and platform owners often need to understand behaviors like how students solve certain questions or where they get stuck and why. However, it is challenging for them to effectively analyze students’ coding data. To this end, we propose BlockLens, a novel visual analytics system to assist instructors and platform owners in analyzing students’ block-based coding behaviors, mistakes, and problem-solving patterns. BlockLens enables the grouping of students by question progress and performance, identification of common problem-solving strategies and pitfalls, and presentation of insights …

Reinforcement Learning Approach To Solve Dynamic Bi-Objective Police Patrol Dispatching And Rescheduling Problem, Waldy Joe, Hoong Chuin Lau, Jonathan Pan

Research Collection School Of Computing and Information Systems

Police patrol aims to fulfill two main objectives namely to project presence and to respond to incidents in a timely manner. Incidents happen dynamically and can disrupt the initially-planned patrol schedules. The key decisions to be made will be which patrol agent to be dispatched to respond to an incident and subsequently how to adapt the patrol schedules in response to such dynamically-occurring incidents whilst still fulfilling both objectives; which sometimes can be conflicting. In this paper, we define this real-world problem as a Dynamic Bi-Objective Police Patrol Dispatching and Rescheduling Problem and propose a solution approach that combines Deep …

Deep Learning For Person Re-Identification: A Survey And Outlook, Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, Steven C. H. Hoi

Research Collection School Of Computing and Information Systems

Person re-identification (Re-ID) aims at retrieving a person of interest across multiple non-overlapping cameras. With the advancement of deep neural networks and increasing demand of intelligent video surveillance, it has gained significantly increased interest in the computer vision community. By dissecting the involved components in developing a person Re-ID system, we categorize it into the closed-world and open-world settings. We first conduct a comprehensive overview with in-depth analysis for closed-world person Re-ID from three different perspectives, including deep feature representation learning, deep metric learning and ranking optimization. With the performance saturation under closed-world setting, the research focus for person Re-ID …

Decomposing Generation Networks With Structure Prediction For Recipe Generation, Hao Wang, Guosheng Lin, Steven C. H. Hoi, Chunyan Miao

Research Collection School Of Computing and Information Systems

Recipe generation from food images and ingredients is a challenging task, which requires the interpretation of the information from another modality. Different from the image captioning task, where the captions usually have one sentence, cooking instructions contain multiple sentences and have obvious structures. To help the model capture the recipe structure and avoid missing some cooking details, we propose a novel framework: Decomposing Generation Networks (DGN) with structure prediction, to get more structured and complete recipe generation outputs. Specifically, we split each cooking instruction into several phases, and assign different sub-generators to each phase. Our approach includes two novel ideas: …

Simultaneous Energy Harvesting And Gait Recognition Using Piezoelectric Energy Harvester, Dong Ma, Guohao Lan, Weitao Xu, Mahbub Hassan, Wen Hu

Research Collection School Of Computing and Information Systems

Piezoelectric energy harvester, which generates electricity from stress or vibrations, is gaining increasing attention as a viable solution to extend battery life in wearables. Recent research further reveals that, besides generating energy, PEH can also serve as a passive sensor to detect human gait power-efficiently because its stress or vibration patterns are significantly influenced by the gait. However, as PEHs are not designed for precise measurement of motion, achievable gait recognition accuracy remains low with conventional classification algorithms. The accuracy deteriorates further when the generated electricity is stored simultaneously. To classify gait reliably while simultaneously storing generated energy, we make …

Deep One-Class Classification Via Interpolated Gaussian Descriptor, Yuanhong Chen, Yu Tian, Guansong Pang, Gustavo Carneiro

Research Collection School Of Computing and Information Systems

One-class classification (OCC) aims to learn an effective data description to enclose all normal training samples and detect anomalies based on the deviation from the data description. Current state-of-the-art OCC models learn a compact normality description by hyper-sphere minimisation, but they often suffer from overfitting the training data, especially when the training set is small or contaminated with anomalous samples. To address this issue, we introduce the interpolated Gaussian descriptor (IGD) method, a novel OCC model that learns a one-class Gaussian anomaly classifier trained with adversarially interpolated training samples. The Gaussian anomaly classifier differentiates the training samples based on their …

Metaformer Is Actually What You Need For Vision, Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xinchao Wang, Jiashi Feng, Shuicheng Yan

Research Collection School Of Computing and Information Systems

Transformers have shown great potential in computer vision tasks. A common belief is their attention-based token mixer module contributes most to their competence. However, recent works show the attention-based module in transformers can be replaced by spatial MLPs and the resulted models still perform quite well. Based on this observation, we hypothesize that the general architecture of the transformers, instead of the specific token mixer module, is more essential to the model's performance. To verify this, we deliberately replace the attention module in transformers with an embarrassingly simple spatial pooling operator to conduct only basic token mixing. Surprisingly, we observe …

Differential Cost Analysis With Simultaneous Potentials And Anti-Potentials, Dorde Zikelic, Bor-Yuh Evan Chang, Pauline Bolignano, Franco Raimondi

Research Collection School Of Computing and Information Systems

We present a novel approach to differential cost analysis that, given a program revision, attempts to statically bound the difference in resource usage, or cost, between the two program versions. Differential cost analysis is particularly interesting because of the many compelling applications for it, such as detecting resource-use regressions at code-review time or proving the absence of certain side-channel vulnerabilities. One prior approach to differential cost analysis is to apply relational reasoning that conceptually constructs a product program on which one can over-approximate the difference in costs between the two program versions. However, a significant challenge in any relational approach …

Do-Gan: A Double Oracle Framework For Generative Adversarial Networks, Aye Phyu Phye Aung, Xinrun Wang, Runsheng Yu, Bo An, Senthilnath Jayavelu, Xiaoli Li

Research Collection School Of Computing and Information Systems

In this paper, we propose a new approach to train Gen-erative Adversarial Networks (GANs) where we deploy a double-oracle framework using the generator and discrim-inator oracles. GAN is essentially a two-player zero-sum game between the generator and the discriminator. Training GANs is challenging as a pure Nash equilibrium may not exist and even finding the mixed Nash equilibrium is difficult as GANs have a large-scale strategy space. In DO-GAN, we extend the double oracle framework to GANs. We first generalize the players' strategies as the trained models of generator and discriminator from the best response or-acles. We then compute the …

Diagnostics Of Dementia From Structural And Functional Markers Of Brain Atrophy With Machine Learning, Tetiana Habuza

Dissertations

Dementia is a condition in which higher mental functions are disrupted. It currently affects an estimated 57 million people throughout the world. A dementia diagnosis is difficult since neither anatomical indicators nor functional testing is currently sufficiently sensitive or specific. There remains a long list of outstanding issues that must be addressed. First, multimodal diagnosis has yet to be introduced into the early stages of dementia screening. Second, there is no accurate instrument for predicting the progression of pre-dementia. Third, non-invasive testing cannot be used to provide differential diagnoses. By creating ML models of normal and accelerated brain aging, we …

Artist-Configurable Node-Based Approach To Generate Procedural Brush Stroke Textures For Digital Painting, Keavon Chambers

Master's Theses

Digital painting is the field of software designed to provide artists a virtual medium to emulate the experience and results of physical drawing. Several hardware and software components come together to form a whole workflow, ranging from the physical input devices, to the stroking process, to the texture content authorship. This thesis explores an artist-friendly approach to synthesize the textures that give life to digital brush strokes.

Most painting software provides a limited library of predefined brush textures. They aim to offer styles approximating physical media like paintbrushes, pencils, markers, and airbrushes. Often these are static bitmap textures that are …

Wildfire Risk Assessment Using Convolutional Neural Networks And Modis Climate Data, Sean F. Nesbit

Master's Theses

Wildfires burn millions of acres of land each year leading to the destruction of homes and wildland ecosystems while costing governments billions in funding. As climate change intensifies drought volatility across the Western United States, wildfires are likely to become increasingly severe. Wildfire risk assessment and hazard maps are currently employed by fire services, but can often be outdated. This paper introduces an image-based dataset using climate and wildfire data from NASA’s Moderate Resolution Imaging Spectroradiometer (MODIS). The dataset consists of 32 climate and topographical layers captured across 0.1 deg by 0.1 deg tiled regions in California and Nevada between …

Out-Of-Core Gpu Path Tracing On Large Instanced Scenes Via Geometry Streaming, Jeremy Berchtold

Master's Theses

We present a technique for out-of-core GPU path tracing of arbitrarily large scenes that is compatible with hardware-accelerated ray-tracing. Our technique improves upon previous works by subdividing the scene spatially into streamable chunks that are loaded using a priority system that maximizes ray throughput and minimizes GPU memory usage. This allows for arbitrarily large scaling of scene complexity. Our system required under 19 minutes to render a solid color version of Disney's Moana Island scene (39.3 million instances, 261.1 million unique quads, and 82.4 billion instanced quads at a resolution of 1024x429 and 1024spp on an RTX 5000 (24GB memory …

A Unified View Of A Human Digital Twin, Michael Miller, Emily Spatz

Faculty Publications

The term human digital twin has recently been applied in many domains, including medical and manufacturing. This term extends the digital twin concept, which has been illustrated to provide enhanced system performance as it combines system models and analyses with real-time measurements for an individual system to improve system maintenance. Human digital twins have the potential to change the practice of human system integration as these systems employ real-time sensing and feedback to tightly couple measurements of human performance, behavior, and environmental influences throughout a product’s life cycle to human models to improve system design and performance. However, as this …

Scenario Acceleration Through Automated Modelling: A Method And System For Creating Traceable Quantitative Future Scenarios Based On Fcm System Modeling And Natural Language Processing, Christopher W.H. Davis

Dissertations and Theses

Scenario planning is used extensively in strategic planning because it helps leaders broaden their perspectives and make better decisions by presenting possible futures in story form. Some of the benefits of using scenarios include breaking away from groupthink, creating better products, acceleration of organization learning and reducing bias. Product development teams, particularly for digital products, are gaining more autonomy in organizations and tend to manage risk by undergoing very short development iterations on their products while leaning on their consumers for feedback -- a process known as agile development. This method tends to limit the perspective of the team and …