Computer Sciences Commons^™

Efficient Conditional Gan Transfer With Knowledge Propagation Across Classes, Shahbazi. Mohamad, Zhiwu Huang, Huang, Danda Pani Paudel, Ajad Chhatkuli, Gool L. Van

Research Collection School Of Computing and Information Systems

Generative adversarial networks (GANs) have shown impressive results in both unconditional and conditional image generation. In recent literature, it is shown that pre-trained GANs, on a different dataset, can be transferred to improve the image generation from a small target data. The same, however, has not been well-studied in the case of conditional GANs (cGANs), which provides new opportunities for knowledge transfer compared to unconditional setup. In particular, the new classes may borrow knowledge from the related old classes, or share knowledge among themselves to improve the training. This motivates us to study the problem of efficient conditional GAN transfer …

Ganmut: Learning Interpretable Conditional Space For A Gamut Of Emotions, S. D'Apolito, D.P. Paundel, Zhiwu Huang, A.R. Vergara, Gool L. Van

Research Collection School Of Computing and Information Systems

Humans can communicate emotions through a plethora of facial expressions, each with its own intensity, nuances and ambiguities. The generation of such variety by means of conditional GANs is limited to the expressions encoded in the used label system. These limitations are caused either due to burdensome labeling demand or the confounded label space. On the other hand, learning from inexpensive and intuitive basic categorical emotion labels leads to limited emotion variability. In this paper, we propose a novel GAN-based framework which learns an expressive and interpretable conditional space (usable as a label space) of emotions, instead of conditioning on …

Reciprocal Transformations For Unsupervised Video Object Segmentation, Sucheng Ren, Wenxi Liu, Yongtuo Liu, Haoxin Chen, Guoqiang Han, Shengfeng He

Research Collection School Of Computing and Information Systems

Unsupervised video object segmentation (UVOS) aims at segmenting the primary objects in videos without any human intervention. Due to the lack of prior knowledge about the primary objects, identifying them from videos is the major challenge of UVOS. Previous methods often regard the moving objects as primary ones and rely on optical flow to capture the motion cues in videos, but the flow information alone is insufficient to distinguish the primary objects from the background objects that move together. This is because, when the noisy motion features are combined with the appearance features, the localization of the primary objects is …

Learning From The Master: Distilling Cross-Modal Advanced Knowledge For Lip Reading, Sucheng Ren, Yong Du, Jianming Lv, Guoqiang Han, Shengfeng He

Research Collection School Of Computing and Information Systems

Lip reading aims to predict the spoken sentences from silent lip videos. Due to the fact that such a vision task usually performs worse than its counterpart speech recognition, one potential scheme is to distill knowledge from a teacher pretrained by audio signals. However, the latent domain gap between the cross-modal data could lead to a learning ambiguity and thus limits the performance of lip reading. In this paper, we propose a novel collaborative framework for lip reading, and two aspects of issues are considered: 1) the teacher should understand bi-modal knowledge to possibly bridge the inherent cross-modal gap; 2) …

Discovering Interpretable Latent Space Directions Of Gans Beyond Binary Attributes, Huiting Yang, Liangyu Chai, Qiang Wen, Shuang Zhao, Zixun Sun, Shengfeng He

Research Collection School Of Computing and Information Systems

Generative adversarial networks (GANs) learn to map noise latent vectors to high- fidelity image outputs. It is found that the input latent space shows semantic correlations with the output image space. Recent works aim to interpret the latent space and discover meaningful directions that correspond to human interpretable image transformations. However, these methods either rely on explicit scores of attributes (e.g., memorability) or are restricted to binary ones (e.g., gender), which largely limits the applicability of editing tasks, especially for free- form artistic tasks like style/anime editing. In this paper, we propose an adversarial method, AdvStyle, for discovering interpretable directions …

Spatially-Invariant Style-Codes Controlled Makeup Transfer, Han Deng, Chu Han, Hongmin Cai, Guoqiang Han, Shengfeng He

Research Collection School Of Computing and Information Systems

Transferring makeup from the misaligned reference image is challenging. Previous methods overcome this barrier by computing pixel-wise correspondences between two images, which is inaccurate and computational-expensive. In this paper, we take a different perspective to break down the makeup transfer problem into a two-step extraction-assignment process. To this end, we propose a Style-based Controllable GAN model that consists of three components, each of which corresponds to target style-code encoding, face identity features extraction, and makeup fusion, respectively. In particular, a Part-specific Style Encoder encodes the component-wise makeup style of the reference image into a style-code in an intermediate latent space …

Soarnet, Deep Learning Thermal Detection For Free Flight, Jake T. Tallman

Master's Theses

Thermals are regions of rising hot air formed on the ground through the warming of the surface by the sun. Thermals are commonly used by birds and glider pilots to extend flight duration, increase cross-country distance, and conserve energy. This kind of powerless flight using natural sources of lift is called soaring. Once a thermal is encountered, the pilot flies in circles to keep within the thermal, so gaining altitude before flying off to the next thermal and towards the destination. A single thermal can net a pilot thousands of feet of elevation gain, however estimating thermal locations is not …

Prediction Of Sleep Quality In Live-Alone Diabetic Seniors Using Unobtrusive In-Home Sensors, Barry Nuqoba, Hwee-Pink Tan

Research Collection School Of Computing and Information Systems

Diabetes, a chronic disease that occurs when the pancreas does not produce enough insulin or when the body cannot effectively utilize its insulin, is increasingly recognized as a significant health burden and affects many older adults. Poor sleep quality in diabetic seniors worsens the diabetes condition, but most seniors are tend to regard poor sleep quality as a usual event and do not seek treatment. This study aims to detect poor sleep quality in diabetic seniors through passive in-home monitoring to inform intervention (e.g., seeking diagnosis and treatment) to improve the physical and mental health of diabetic seniors. We derive …

Virtual Reality For Hazard Mitigation And Community Resilience: An Interdisciplinary Collaboration With Community Engagement To Enhance Risk Awareness, N. J. Stone, G. Yan, Fiona Fui-Hoon Nah, C. Sabharwal, K. Angle, F. E. Hatch, S. Runnels, V. Brown, G. M. Schoor, C. Engelbrecht

Research Collection School Of Computing and Information Systems

To achieve community resilience and mitigate the consequences of natural hazards, community officials must balance competing priorities for local resources and funding. Besides the challenge of dealing with multiple competing priorities, community officials face another challenge: low risk awareness of natural hazards by the public and other stakeholders. Considering that virtual reality (VR) has been used to enhance learning and to change attitudes and behaviors, animating natural hazards in VR has the potential to enhance stakeholders’ (e.g., the public, local/state/federal governments, insurance agencies, and property owners) risk awareness. Informed stakeholders make better decisions related to protective action. Therefore, we propose …

Impact Of User Traversal On Performance Of Stem Learners In Immersive Virtual Environments, Eric W. Nersesian

Dissertations

The emerging technologies of augmented and virtual reality (AR/VR) may have vast implications to societal communication and representation of information. AR/VR computer interfaces are unique in that they may be placed spatially around the user in three-dimensional (3D) space; this affords new methods of both presentation and user interaction with the target information.

This may be especially impactful in the education of science, technology, engineering, and mathematics (STEM) professionals. Prior research has shown that simulations and visualizations improve the performance of STEM learners compared to live instruction and textbook reading. Yet, research into AR/VR as a learning environment for widespread …

Translating Natural Language Queries To Sparql, Shreya Satish Bhajikhaye

Master's Projects

The Semantic Web is an extensive knowledge base that contains facts in the form of RDF
triples. These facts are not easily accessible to the average user because to use them requires
an understanding of ontologies and a query language like SPARQL. Question answering systems
form a layer of abstraction on linked data to overcome these issues. These systems allow the
user to input a question in a natural language and receive the equivalent SPARQL query. The
user can then execute the query on the database to fetch the desired results. The standard
techniques involved in translating natural language questions …

Using Oracle To Solve Zookeeper On Two-Replica Problems, Ching-Chan Lee

Master's Projects

The project introduces an Oracle, a failure detector, in Apache ZooKeeper and makes it fault-tolerant in a two-node system. The project demonstrates the Oracle authorizes the primary process to maintain the liveness when the majority’s rule becomes an obstacle to continue Apache ZooKeeper service. In addition to the property of accuracy and completeness from Chandra et al.’s research, the project proposes the property of see to avoid losing transactions and the property of mutual exclusion to avoid split-brain issues. The hybrid properties render not only more sounder flexibility in the implementation but also stronger guarantees on safety. Thus, the Oracle …

Federated Learning In Gaze Recognition (Fligr), Arun Gopal Govindaswamy

College of Computing and Digital Media Dissertations

The efficiency and generalizability of a deep learning model is based on the amount and diversity of training data. Although huge amounts of data are being collected, these data are not stored in centralized servers for further data processing. It is often infeasible to collect and share data in centralized servers due to various medical data regulations. This need for diversely distributed data and infeasible storage solutions calls for Federated Learning (FL). FL is a clever way of utilizing privately stored data in model building without the need for data sharing. The idea is to train several different models locally …

Pitcher Effectiveness: A Step Forward For In Game Analytics And Pitcher Evaluation, Christopher Watkins, Vincent Berardi, Cyril Rakovski

Mathematics, Physics, and Computer Science Faculty Articles and Research

With the introduction of Statcast in 2015, baseball analytics have become more precise. Statcast allows every play to be accurately tracked and the data it generates is easily accessible through Baseball Savant, which opens the opportunity for improved performance statistics to be developed. In this paper we propose a new tool, Pitcher Effectiveness, that uses Statcast data to evaluate starting pitchers dynamically, based on the results of in-game outcomes after each pitch. Pitcher Effectiveness successfully predicts instances where starting pitchers give up several runs, which we believe make it a new and important tool for the in-game and post-game evaluation …

Analysis Of Theoretical And Applied Machine Learning Models For Network Intrusion Detection, Jonah Baron

Masters Theses & Doctoral Dissertations

Network Intrusion Detection System (IDS) devices play a crucial role in the realm of network security. These systems generate alerts for security analysts by performing signature-based and anomaly-based detection on malicious network traffic. However, there are several challenges when configuring and fine-tuning these IDS devices for high accuracy and precision. Machine learning utilizes a variety of algorithms and unique dataset input to generate models for effective classification. These machine learning techniques can be applied to IDS devices to classify and filter anomalous network traffic. This combination of machine learning and network security provides improved automated network defense by developing highly-optimized …

Automatic Solution Summarization For Crash Bugs, Haoye Wang, Xin Xia, David Lo, John C. Grundy, Xinyu Wang

Research Collection School Of Computing and Information Systems

The causes of software crashes can be hidden anywhere in the source code and development environment. When encountering software crashes, recurring bugs that are discussed on Q&A sites could provide developers with solutions to their crashing problems. However, it is difficult for developers to accurately search for relevant content on search engines, and developers have to spend a lot of manual effort to find the right solution from the returned results. In this paper, we present CRASOLVER, an approach that takes into account both the structural information of crash traces and the knowledge of crash-causing bugs to automatically summarize solutions …

On The Root Of Trust Identification Problem, Ivan De Oliveira Nunes, Xuhua Ding, Gene Tsudik

Research Collection School Of Computing and Information Systems

Trusted Execution Environments (TEEs) are becoming ubiquitous and are currently used in many security applications: from personal IoT gadgets to banking and databases. Prominent examples of such architectures are Intel SGX, ARM TrustZone, and Trusted Platform Modules (TPMs). A typical TEE relies on a dynamic Root of Trust (RoT) to provide security services such as code/data confidentiality and integrity, isolated secure software execution, remote attestation, and sensor auditing. Despite their usefulness, there is currently no secure means to determine whether a given security service or task is being performed by the particular RoT within a specific physical device. We refer …

Smart Contract Security: A Practitioners' Perspective, Zhiyuan Wan, Xin Xia, David Lo, Jiachi Chen, Xiapu Luo, Xiaohu Yang

Research Collection School Of Computing and Information Systems

Smart contracts have been plagued by security incidents, which resulted in substantial financial losses. Given numerous research efforts in addressing the security issues of smart contracts, we wondered how software practitioners build security into smart contracts in practice. We performed a mixture of qualitative and quantitative studies with 13 interviewees and 156 survey respondents from 35 countries across six continents to understand practitioners' perceptions and practices on smart contract security. Our study uncovers practitioners' motivations and deterrents of smart contract security, as well as how security efforts and strategies fit into the development lifecycle. We also find that blockchain platforms …

Learning Index Policies For Restless Bandits With Application To Maternal Healthcare, Arpita Biswas, Gaurav Aggarwal, Pradeep Varakantham, Milind Tambe

Research Collection School Of Computing and Information Systems

In many community health settings, it is crucial to have a systematic monitoring and intervention process to ensure that the patients adhere to healthcare programs, such as periodic health checks or taking medications. When these interventions are expensive, they can be provided to only a fixed small fraction of the patients at any period of time. Hence, it is important to carefully choose the beneficiaries who should be provided with interventions and when. We model this scenario as a restless multi-armed bandit (RMAB) problem, where each beneficiary is assumed to transition from one state to another depending on the intervention …

An Empirical Study Of The Landscape Of Open Source Projects In Baidu, Alibaba, And Tencent, Junxiao Han, Shuiguang Deng, David Lo, Chen Zhi, Jianwei Yin, Xin Xia

Research Collection School Of Computing and Information Systems

Open source software has drawn more and more attention from researchers, developers and companies nowadays. Meanwhile, many Chinese technology companies are embracing open source and choosing to open source their projects. Nevertheless, most previous studies are concentrated on international companies such as Microsoft or Google, while the practical values of open source projects of Chinese technology companies remain unclear. To address this issue, we conduct a mixed-method study to investigate the landscape of projects open sourced by three large Chinese technology companies, namely Baidu, Alibaba, and Tencent (BAT). We study the categories and characteristics of open source projects, the developer's …

A Differential Testing Approach For Evaluating Abstract Syntax Tree Mapping Algorithms, Yuanrui Fan, Xin Xia, David Lo, Ahmed E. Hassan, Yuan Wang, Shanping Li

Research Collection School Of Computing and Information Systems

Abstract syntax tree (AST) mapping algorithms are widely used to analyze changes in source code. Despite the foundational role of AST mapping algorithms, little effort has been made to evaluate the accuracy of AST mapping algorithms, i.e., the extent to which an algorithm captures the evolution of code. We observe that a program element often has only one best-mapped program element. Based on this observation, we propose a hierarchical approach to automatically compare the similarity of mapped statements and tokens by different algorithms. By performing the comparison, we determine if eachof the compared algorithms generates inaccurate mappings for a statement …

Unveiling The Mystery Of Api Evolution In Deep Learning Frameworks: A Case Study Of Tensorflow 2, Zejun Zhang, Yanming Yang, Xin Xia, David Lo, Xiaoxue Ren, John C. Grundy

Research Collection School Of Computing and Information Systems

API developers have been working hard to evolve APIs to provide more simple, powerful, and robust API libraries. Although API evolution has been studied for multiple domains, such as Web and Android development, API evolution for deep learning frameworks has not yet been studied. It is not very clear how and why APIs evolve in deep learning frameworks, and yet these are being more and more heavily used in industry. To fill this gap, we conduct a large-scale and in-depth study on the API evolution of Tensorflow 2, which is currently the most popular deep learning framework. We first extract …

Action Selection For Composable Modular Deep Reinforcement Learning, Vaibhav Gupta, Daksh Anand, Praveen Paruchuri, Akshat Kumar

Research Collection School Of Computing and Information Systems

In modular reinforcement learning (MRL), a complex decision making problem is decomposed into multiple simpler subproblems each solved by a separate module. Often, these subproblems have conflicting goals, and incomparable reward scales. A composable decision making architecture requires that even the modules authored separately with possibly misaligned reward scales can be combined coherently. An arbitrator should consider different module's action preferences to learn effective global action selection. We present a novel framework called GRACIAS that assigns fine-grained importance to the different modules based on their relevance in a given state, and enables composable decision making based on modern deep RL …

Approximate Difference Rewards For Scalable Multigent Reinforcement Learning, Arambam James Singh, Akshat Kumar

Research Collection School Of Computing and Information Systems

We address the problem of multiagent credit assignment in a large scale multiagent system. Difference rewards (DRs) are an effective tool to tackle this problem, but their exact computation is known to be challenging even for small number of agents. We propose a scalable method to compute difference rewards based on aggregate information in a multiagent system with large number of agents by exploiting the symmetry present in several practical applications. Empirical evaluation on two multiagent domains—air-traffic control and cooperative navigation, shows better solution quality than previous approaches.

Guest Editorial: Non-Iid Outlier Detection In Complex Contexts, Guansong Pang, Fabrizio Angiulli, Mihai Cucuringu, Huan Liu

Research Collection School Of Computing and Information Systems

Outlier detection, also known as anomaly detection, aims at identifying data instances that are rare or significantly different from the majority of instances. Due to its significance in many critical domains like cybersecurity, fintech, healthcare, public security, and AI safety, outlier detection has been one of the most active research areas in various communities, such as machine learning, data mining, computer vision, and statistics. Traditional outlier-detection techniques generally assume that data are independent and identically distributed (IID), which are significantly challenged in complex contexts where data are actually non-IID. These contexts are ubiquitous in not only graph data, sequence data, …

Tripdecoder: Study Travel Time Attributes And Route Preferences Of Metro Systems From Smart Card Data, Xiancai Tian, Baihua Zheng, Yazhe Wang, Hsao-Ting Huang, Chih-Cheng Hung

Research Collection School Of Computing and Information Systems

In this paper, we target at recovering the exact routes taken by commuters inside a metro system that are not captured by an Automated Fare Collection (AFC) system and hence remain unknown. We strategically propose two inference tasks to handle the recovering, one to infer the travel time of each travel link that contributes to the total duration of any trip inside a metro network and the other to infer the route preferences based on historical trip records and the travel time of each travel link inferred in the previous inference task. As these two inference tasks have interrelationship, most …

How Do Software Developers Use Github Actions To Automate Their Workflows?, Timothy Kinsman, Mairieli Wessel, Marco Gerosa, Christoph Treude

Research Collection School Of Computing and Information Systems

Automated tools are frequently used in social coding repositories to perform repetitive activities that are part of the distributed software development process. Recently, GitHub introduced GitHub Actions, a feature providing automated work-flows for repository maintainers. Although several Actions have been built and used by practitioners, relatively little has been done to evaluate them. Understanding and anticipating the effects of adopting such kind of technology is important for planning and management. Our research is the first to investigate how developers use Actions and how several activity indicators change after their adoption. Our results indicate that, although only a small subset of …

Tensor Low-Rank Representation For Data Recovery And Clustering, Pan Zhou, Canyi Lu, Jiashi Feng, Zhouchen Lin, Shuicheng Yan

Research Collection School Of Computing and Information Systems

Multi-way or tensor data analysis has attracted increasing attention recently, with many important applications in practice. This article develops a tensor low-rank representation (TLRR) method, which is the first approach that can exactly recover the clean data of intrinsic low-rank structure and accurately cluster them as well, with provable performance guarantees. In particular, for tensor data with arbitrary sparse corruptions, TLRR can exactly recover the clean data under mild conditions; meanwhile TLRR can exactly verify their true origin tensor subspaces and hence cluster them accurately. TLRR objective function can be optimized via efficient convex programing with convergence guarantees. Besides, we …

Digital Banking Accelerator: A Service-Oriented Architecture Starter Kit For Banks, Alan @ Ali Madjelisi Megargel, Shankararaman, Venky

Research Collection School Of Computing and Information Systems

Digital banking refers to the delivery of interactive financial services through online mechanisms which include web and mobile apps. The main barrier to digital banking for traditional banks, is the presence of legacy core banking systems. Service Oriented Architecture (SOA) is a key enabler to overcome this barrier, and a bank’s level of SOA maturity influences its time-to-market capability of delivering new innovative digital banking solutions. However, most traditional banks struggle with implementing an SOA due to a number of technology and organizational challenges, and the overall steep learning curve. This paper proposes a Digital Banking Accelerator, a “starter kit” …

Solving 3d Bin Packing Problem Via Multimodal Deep Reinforcement Learning, Yuan Jiang, Zhiguang Cao, Jie Zhang

Research Collection School Of Computing and Information Systems

Recently, there is growing attention on applying deep reinforcement learning (DRL) to solve the 3D bin packing problem (3D BPP), given its favorable generalization and independence of ground-truth label. However, due to the relatively less informative yet computationally heavy encoder, and considerably large action space inherent to the 3D BPP, existing methods are only able to handle up to 50 boxes. In this paper, we propose to alleviate this issue via an end-to-end multimodal DRL agent, which sequentially addresses three sub-tasks of sequence, orientation and position, respectively. The resulting architecture enables the agent to solve large-scale instances of 100 boxes …