Graphics and Human Computer Interfaces Commons^™

Contrastive Learning For Unsupervised Auditory Texture Models, Christina Trexler

Computer Science and Computer Engineering Undergraduate Honors Theses

Sounds with a high level of stationarity, also known as sound textures, have perceptually relevant features which can be captured by stimulus-computable models. This makes texture-like sounds, such as those made by rain, wind, and fire, an appealing test case for understanding the underlying mechanisms of auditory recognition. Previous auditory texture models typically measured statistics from auditory filter bank representations, and the statistics they used were somewhat ad-hoc, hand-engineered through a process of trial and error. Here, we investigate whether a better auditory texture representation can be obtained via contrastive learning, taking advantage of the stationarity of auditory textures to …

Self-Supervised Multi-Class Pre-Training For Unsupervised Anomaly Detection And Segmentation In Medical Images, Yu Tian, Fengbei Liu, Guansong Pang, Yuanhong Chen, Yuyuan Liu, Johan W. Verjans, Rajvinder Singh

Research Collection School Of Computing and Information Systems

Unsupervised anomaly detection (UAD) that requires only normal (healthy) training images is an important tool for enabling the development of medical image analysis (MIA) applications, such as disease screening, since it is often difficult to collect and annotate abnormal (or disease) images in MIA. However, heavily relying on the normal images may cause the model training to overfit the normal class. Self-supervised pre-training is an effective solution to this problem. Unfortunately, current self-supervision methods adapted from computer vision are sub-optimal for MIA applications because they do not explore MIA domain knowledge for designing the pretext tasks or the training process. …

Conquer: Contextual Query-Aware Ranking For Video Corpus Moment Retrieval, Zhijian Hou, Chong-Wah Ngo, W. K. Chan

Research Collection School Of Computing and Information Systems

This paper tackles a recently proposed Video Corpus Moment Retrieval task. This task is essential because advanced video retrieval applications should enable users to retrieve a precise moment from a large video corpus. We propose a novel CONtextual QUery-awarE Ranking (CONQUER) model for effective moment localization and ranking. CONQUER explores query context for multi-modal fusion and representation learning in two different steps. The first step derives fusion weights for the adaptive combination of multi-modal video content. The second step performs bi-directional attention to tightly couple video and query as a single joint representation for moment localization. As query context is …

Constrained Contrastive Distribution Learning For Unsupervised Anomaly Detection And Localisation In Medical Images, Yu Tian, Guansong Pang, Fengbei Liu, Yuanhong Chen, Seon Ho Shin, Johan W. Verjans, Rajvinder Singh

Research Collection School Of Computing and Information Systems

Unsupervised anomaly detection (UAD) learns one-class classifiers exclusively with normal (i.e., healthy) images to detect any abnormal (i.e., unhealthy) samples that do not conform to the expected normal patterns. UAD has two main advantages over its fully supervised counterpart. Firstly, it is able to directly leverage large datasets available from health screening programs that contain mostly normal image samples, avoiding the costly manual labelling of abnormal samples and the subsequent issues involved in training with extremely class-imbalanced data. Further, UAD approaches can potentially detect and localise any type of lesions that deviate from the normal patterns. One significant challenge faced …

Fine-Grained Analysis Of Structured Output Prediction, Waleed Mustafa, Yunwen Lei, Antoine Ledent, Marius And Kloft

Research Collection School Of Computing and Information Systems

In machine learning we often encounter structured output prediction problems (SOPPs), i.e. problems where the output space admits a rich internal structure. Application domains where SOPPs naturally occur include natural language processing, speech recognition, and computer vision. Typical SOPPs have an extremely large label set, which grows exponentially as a function of the size of the output. Existing generalization analysis implies generalization bounds with at least a square-root dependency on the cardinality d of the label set, which can be vacuous in practice. In this paper, we significantly improve the state of the art by developing novel high-probability bounds with …

Learning Interpretable Concept Groups In Cnns, Saurabh Varshneya, Antoine Ledent, Rob Vandermeulen, Yunwen Lei, Matthias Enders, Damian Borth, Marius Kloft

Research Collection School Of Computing and Information Systems

We propose a novel training methodology---Concept Group Learning (CGL)---that encourages training of interpretable CNN filters by partitioning filters in each layer into concept groups, each of which is trained to learn a single visual concept. We achieve this through a novel regularization strategy that forces filters in the same group to be active in similar image regions for a given layer. We additionally use a regularizer to encourage a sparse weighting of the concept groups in each layer so that a few concept groups can have greater importance than others. We quantitatively evaluate CGL's model interpretability using standard interpretability evaluation …

Computational Frameworks For Multi-Robot Cooperative 3d Printing And Planning, Laxmi Prasad Poudel

Graduate Theses and Dissertations

This dissertation proposes a novel cooperative 3D printing (C3DP) approach for multi-robot additive manufacturing (AM) and presents scheduling and planning strategies that enable multi-robot cooperation in the manufacturing environment. C3DP is the first step towards achieving the overarching goal of swarm manufacturing (SM). SM is a paradigm for distributed manufacturing that envisions networks of micro-factories, each of which employs thousands of mobile robots that can manufacture different products on demand. SM breaks down the complicated supply chain used to deliver a product from a large production facility from one part of the world to another. Instead, it establishes a network …

Design And Development Of Techniques To Ensure Integrity In Fog Computing Based Databases, Abdulwahab Fahad S. Alazeb

Graduate Theses and Dissertations

The advancement of information technology in coming years will bring significant changes to the way sensitive data is processed. But the volume of generated data is rapidly growing worldwide. Technologies such as cloud computing, fog computing, and the Internet of things (IoT) will offer business service providers and consumers opportunities to obtain effective and efficient services as well as enhance their experiences and services; increased availability and higher-quality services via real-time data processing augment the potential for technology to add value to everyday experiences. This improves human life quality and easiness. As promising as these technological innovations, they are prone …

How Important Is The Train-Validation Split In Meta-Learning?, Yu Bai, Minshuo Chen, Pan Zhou, Tuo Zhao, D. Jason Lee, Sham Kakade, Huan Wang, Caiming Xiong

Research Collection School Of Computing and Information Systems

Meta-learning aims to perform fast adaptation on a new task through learning a “prior” from multiple existing tasks. A common practice in meta-learning is to perform a train-validation split (train-val method) where the prior adapts to the task on one split of the data, and the resulting predictor is evaluated on another split. Despite its prevalence, the importance of the train-validation split is not well understood either in theory or in practice, particularly in comparison to the more direct train-train method, which uses all the pertask data for both training and evaluation. We provide a detailed theoretical study on whether …

Projecting Your View Attentively: Monocular Road Scene Layout Estimation Via Cross-View Transformation, Weixiang Yang, Qi Li, Wenxi Liu, Yuanlong Yu, Yuexin Ma, Shengfeng He, Jia Pan

Research Collection School Of Computing and Information Systems

HD map reconstruction is crucial for autonomous driving. LiDAR-based methods are limited due to the deployed expensive sensors and time-consuming computation. Camera-based methods usually need to separately perform road segmentation and view transformation, which often causes distortion and the absence of content. To push the limits of the technology, we present a novel framework that enables reconstructing a local map formed by road layout and vehicle occupancy in the bird's-eye view given a front-view monocular image only. In particular, we propose a cross-view transformation module, which takes the constraint of cycle consistency between views into account and makes full use …

Engaging Drivers Via Competition: A Case Study With Arena, Hao Cheng, Shuyu Wei, Lingyu Zhang, Zimu Zhou, Yongxin. Tong

Research Collection School Of Computing and Information Systems

Sustained work enthusiasms of drivers are crucial for the success of large-scale ride-hailing platforms. In this paper, we conduct the first-of-its-kind exploration to encourage active participation of drivers via competition. We design Arena, a competition where drivers compete for prizes via completing more trips. Through a pilot study covering over 2,600 participants, we uncover the easy-win problem, an overlooked and serious issue in competition design for real-world drivers. It refers to situations where one competitor does not show up during competition whereas the other easily wins. To solve the easy-win problem without impairing motivation of drivers, we devise a novel …

When Program Analysis Meets Bytecode Search: Targeted And Efficient Inter-Procedural Analysis Of Modern Android Apps In Backdroid, Daoyuan Wu, Debin Gao, Robert H. Deng, Rocky Chang

Research Collection School Of Computing and Information Systems

Widely-used Android static program analysis tools,e.g., Amandroid and FlowDroid, perform the whole-app interprocedural analysis that is comprehensive but fundamentallydifficult to handle modern (large) apps. The average app size hasincreased three to four times over five years. In this paper, weexplore a new paradigm of targeted inter-procedural analysis thatcan skip irrelevant code and focus only on the flows of securitysensitive sink APIs. To this end, we propose a technique calledon-the-fly bytecode search, which searches the disassembled appbytecode text just in time when a caller needs to be located. In thisway, it guides targeted (and backward) inter-procedural analysisstep by step until reaching …

Learning Contextual Causality Between Daily Events From Time-Consecutive Images, Hongming Zhang, Yintong Huo, Xinran Zhao, Yangqiu Song, Dan Roth

Research Collection School Of Computing and Information Systems

Conventional textual-based causal knowledge acquisition methods typically require laborious and expensive human annotations. As a result, their scale is often limited. Moreover, as no context is provided during the annotation, the resulting causal knowledge records (e.g., ConceptNet) typically do not consider the context. In this paper, we move out of the textual domain to explore a more scalable way of acquiring causal knowledge and investigate the possibility of learning contextual causality from the visual signal. Specifically, we first propose a high-quality dataset Vis-Causal and then conduct experiments to demonstrate that with good language and visual representations, it is possible to …

A Hybrid Gaze Pointer With Voice Control, Indhuja Ravi

Master's Projects

Accessibility in technology has been a challenge since the beginning of the 1800s. Starting with building typewriters for the blind by Pellegrino Turri to the on-screen keyboard built by Microsoft, there have been several advancements towards assistive technologies. The basic tools necessary for anyone to operate a computer are to be able to navigate the device, input information, and perceive the output. All these three categories have been undergoing tremendous advancements over the years. Especially, with the internet boom, it has now become a necessity to point onto a computer screen. This has somewhat attracted research into this particular area. …

Learning Intermediate Representations For Question Answering Systems, Zakery T. Clarke

Computer Science ETDs

Question answering systems are models that can perform natural language processing (NLP) on a question, retrieve an answer from a datasource, and communicate it to a user. In question answering systems, it is important for the system to learn an underlying representation for a piece of text. There are many systems that have achieved incredible accuracy on question answering datasets such as the Stanford Question and Answer Dataset (SQuAD), but these systems often encode their knowledge in a manner that is impossible to verify. Many current models would benefit more from verifiability, than marginal accuracy improvements.

We propose a method …

City Goers: An Exploration Into Creating Seemingly Intelligent A.I. Systems, Matthew Brooke

Computer Science and Computer Engineering Undergraduate Honors Theses

Artificial Intelligence systems have come a long way over the years. One particular application of A.I. is its incorporation in video games. A key goal of creating an A.I. system in a video game is to convey a level of intellect to the player. During playtests for Halo: Combat Evolved, the developers at Bungie noticed that players deemed tougher enemies as more intelligent than weaker ones, despite the fact that there were no differences in behavior in the enemies. The tougher enemies provided a greater illusion of intelligence to the players. Inspired by this, I set out to create a …

Brave New World Reboot: Technology’S Role In Consumer Manipulation And Implications For Privacy And Transparency, Allie Mertensotto

Marketing Undergraduate Honors Theses

Most consumers are aware that our data is being obtained and collected through the use of our devices we keep in our homes or even on our person throughout the day. But, it is understated how much data is being collected. Conversations you have with your peers – in a close proximity of a device – are being used to tailor advertising. The advertisements you receive on your devices are uniquely catered to your individual person, due to the fact it consistently uses our data to produce efficient and personal ads. On the flip side, our government is also tapping …

Edgeduet: Tiling Small Object Detection For Edge Assisted Autonomous Mobile Vision, Xu Wang, Zheng Yang, Jiahang Wu, Yi Zhao, Zimu Zhou

Research Collection School Of Computing and Information Systems

Accurate, real-time object detection on resource-constrained devices enables autonomous mobile vision applications such as traffic surveillance, situational awareness, and safety inspection, where it is crucial to detect both small and large objects in crowded scenes. Prior studies either perform object detection locally on-board or offload the task to the edge/cloud. Local object detection yields low accuracy on small objects since it operates on low-resolution videos to fit in mobile memory. Offloaded object detection incurs high latency due to uploading high-resolution videos to the edge/cloud. Rather than either pure local processing or offloading, we propose to detect large objects locally while …

Analog Spiking Neural Network Implementing Spike Timing-Dependent Plasticity On 65 Nm Cmos, Luke Vincent

Graduate Theses and Dissertations

Machine learning is a rapidly accelerating tool and technology used for countless applications in the modern world. There are many digital algorithms to deploy a machine learning program, but the most advanced and well-known algorithm is the artificial neural network (ANN). While ANNs demonstrate impressive reinforcement learning behaviors, they require large power consumption to operate. Therefore, an analog spiking neural network (SNN) implementing spike timing-dependent plasticity is proposed, developed, and tested to demonstrate equivalent learning abilities with fractional power consumption compared to its digital adversary.

Exploring Ai And Multiplayer In Java, Ronni Kurtzhals

Student Academic Conference

I conducted research into three topics: artificial intelligence, package deployment, and multiplayer servers in Java. This research came together to form my project presentation on the implementation of these topics, which I felt accurately demonstrated the various things I have learned from my courses at Moorhead State University. Several resources were consulted throughout the project, including the work of W3Schools and StackOverflow as well as relevant assignments and textbooks from previous classes. I found this project relevant to computer science and information systems for several reasons, such as the AI component and use of SQL data tables; but it was …

Learning And Simulation Algorithms For Constraint Physical Systems, Shuqi Yang

Dartmouth College Master’s Theses

This thesis explores two computational approaches to learn and simulate complex physical systems exhibiting constraint characteristics. The target applications encompass both solids and fluids. On the solid side, we proposed a new family of data-driven simulators to predict the behaviors of an unknown physical system by learning its underpinning constraints. We devised a neural projection operator facilitated by an embedded recursive neural network to interactively enforce the learned underpinning constraints and to predict its various physical behaviors. Our method can automatically uncover a broad range of constraints from observation point data, such as length, angle, bending, collision, boundary effects, and …

Adversarial Meta Sampling For Multilingual Low-Resource Speech Recognition, Yubei Xiao, Ke Gong, Pan Zhou, Guolin Zheng, Xiaodan Liang, Liang Lin

Research Collection School Of Computing and Information Systems

Human doctors with well-structured medical knowledge can diagnose a disease merely via a few conversations with patients about symptoms. In contrast, existing knowledgegrounded dialogue systems often require a large number of dialogue instances to learn as they fail to capture the correlations between different diseases and neglect the diagnostic experience shared among them. To address this issue, we propose a more natural and practical paradigm, i.e., low-resource medical dialogue generation, which can transfer the diagnostic experience from source diseases to target ones with a handful of data for adaptation. It is capitalized on a commonsense knowledge graph to characterize the …

"Who Can Help Me?'': Knowledge Infused Matching Of Support Seekers And Support Providers During Covid-19 On Reddit, Manas Gaur, Kaushik Roy, Aditya Sharma, Biplav Srivastava, Amit Sheth

Publications

During the ongoing COVID-19 crisis, subreddits on Reddit, such as r/Coronavirus saw a rapid growth in user's requests for help (support seekers - SSs) including individuals with varying professions and experiences with diverse perspectives on care (support providers - SPs). Currently, knowledgeable human moderators match an SS with a user with relevant experience, i.e, an SP on these subreddits. This unscalable process defers timely care. We present a medical knowledge-infused approach to efficient matching of SS and SPs validated by experts for the users affected by anxiety and depression, in the context of with COVID-19. After matching, each SP to …

Converting Optical Videos To Infrared Videos Using Attention Gan And Its Impact On Target Detection And Classification Performance, Mohammad Shahab Uddin, Reshad Hoque, Kazi Aminul Islam, Chiman Kwan, David Gribben, Jiang Li

Electrical & Computer Engineering Faculty Publications

To apply powerful deep-learning-based algorithms for object detection and classification in infrared videos, it is necessary to have more training data in order to build high-performance models. However, in many surveillance applications, one can have a lot more optical videos than infrared videos. This lack of IR video datasets can be mitigated if optical-to-infrared video conversion is possible. In this paper, we present a new approach for converting optical videos to infrared videos using deep learning. The basic idea is to focus on target areas using attention generative adversarial network (attention GAN), which will preserve the fidelity of target areas. …

3d Dental Biometrics: Automatic Pose-Invariant Dental Arch Extraction And Matching, Xin Zhong, Zhiyuan Zhang

Research Collection School Of Computing and Information Systems

A novel automatic pose-invariant dental arch extraction and matching framework is developed for 3D dental identification using laser-scanned dental plasters. In our previous attempt [1-5], 3D point-based algorithms have been developed and they have shown a few advantages over existing 2D dental identifications. This study is a continuous effort in developing arch-based algorithms to extract and match dental arch feature in an automatic and pose-invariant way. As best as we know, this is the first attempt at automatic dental arch extraction and matching for 3D dental identification. A Radial Ray Algorithm (RRA) is proposed by projecting dental arch shape from …

Virtual Tutor Personality In Computer Assisted Language Learning, Johanna Dobbriner, Cathy Ennis, Robert J. Ross

Conference papers

The use of intelligent virtual agents in language learning has increased in recent years. Studies into several aspects of personalisation aiming to increase user engagement are an ongoing research topic with avatar personality being one such aspect. As a step towards our development of intelligent virtual avatars, we present two of our initial experiments to explore differences in user interaction with two contrasting avatar personalities -- P1: open-minded, friendly and sociable and P2: closed-off, curt and distant. Each user interacted with a single personality in a video-call setting and gave feedback on the interaction. Our expectations, that P1 would be …

Light Field Compression And Manipulation Via Residual Convolutional Neural Network, Eisa Hedayati

Dissertations, Master's Theses and Master's Reports

Light field (LF) imaging has gained significant attention due to its recent success in microscopy, 3-dimensional (3D) displaying and rendering, augmented and virtual reality usage. Postprocessing of LF enables us to extract more information from a scene compared to traditional cameras. However, the use of LF is still a research novelty because of the current limitations in capturing high-resolution LF in all of its four dimensions. While researchers are actively improving methods of capturing high-resolution LF's, using simulation, it is possible to explore a high-quality captured LF's properties. The immediate concerns following the LF capture are its storage and processing …

Detecting Surface Interactions Via A Wearable Microphone To Improve Augmented Reality Text Entry, R. Habibi

Dissertations, Master's Theses and Master's Reports

This thesis investigates whether we can detect and distinguish between surface interaction events such as tapping or swiping using a wearable mic from a surface. Also, what are the advantages of new text entry methods such as tapping with two fingers simultaneously to enter capital letters and punctuation? For this purpose, we conducted a remote study to collect audio and video of three different ways people might interact with a surface. We also built a CNN classifier to detect taps. Our results show that we can detect and distinguish between surface interaction events such as tap or swipe via a …

A Study Of Multi-Task And Region-Wise Deep Learning For Food Ingredient Recognition, Jingjing Chen, Bin Zhu, Chong-Wah Ngo, Tat-Seng Chua, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Food recognition has captured numerous research attention for its importance for health-related applications. The existing approaches mostly focus on the categorization of food according to dish names, while ignoring the underlying ingredient composition. In reality, two dishes with the same name do not necessarily share the exact list of ingredients. Therefore, the dishes under the same food category are not mandatorily equal in nutrition content. Nevertheless, due to limited datasets available with ingredient labels, the problem of ingredient recognition is often overlooked. Furthermore, as the number of ingredients is expected to be much less than the number of food categories, …

Global Context Aware Convolutions For 3d Point Cloud Understanding, Zhiyuan Zhang, Binh-Son Hua, Wei Chen, Yibin Tian, Sai-Kit Yeung

Research Collection School Of Computing and Information Systems

Recent advances in deep learning for 3D point clouds have shown great promises in scene understanding tasks thanks to the introduction of convolution operators to consume 3D point clouds directly in a neural network. Point cloud data, however, could have arbitrary rotations, especially those acquired from 3D scanning. Recent works show that it is possible to design point cloud convolutions with rotation invariance property, but such methods generally do not perform as well as translation-invariant only convolution. We found that a key reason is that compared to point coordinates, rotation-invariant features consumed by point cloud convolution are not as distinctive. …