Artificial Intelligence and Robotics Commons^™

Introduction, Michael Paulus

SPU Works

Artificial intelligence is rapidly and radically changing our lives and world. This book is a multidisciplinary engagement with the present and future impacts of AI from the standpoint of Christian faith. It provides technological, philosophical, and theological foundations for thinking about AI, as well as a series of reflections on the impact of AI on relationships, behavior, education, work, and moral action. The book serves as an accessible introduction to AI as well as a guide to wise consideration, design, and use of AI by examining foundational understandings and beliefs from a Christian perspective.

Training Thinner And Deeper Neural Networks: Jumpstart Regularization, Carles Riera, Camilo Rey, Thiago Serra, Eloi Puertas, Oriol Pujol

Faculty Conference Papers and Presentations

Neural networks are more expressive when they have multiple layers. In turn, conventional training methods are only successful if the depth does not lead to numerical issues such as exploding or vanishing gradients, which occur less frequently when the layers are sufficiently wide. However, increasing width to attain greater depth entails the use of heavier computational resources and leads to overparameterized models. These subsequent issues have been partially addressed by model compression methods such as quantization and pruning, some of which relying on normalization-based regularization of the loss function to make the effect of most parameters negligible. In this work, …

Monofacial Vs Bifacial Solar Photovoltaic Systems In Snowy Environments, Koami Soulemane Hayibo, Aliaksei Petsiuk, Pierce Mayville, Laura Brown, Joshua M. Pearce

Electrical and Computer Engineering Publications

There has been a recent surge in interest in the more accurate snow loss estimates for solar photovoltaic (PV) systems as large-scale deployments move into northern latitudes. Preliminary results show bifacial modules may clear snow faster than monofacial PV. This study analyzes snow losses on these two types of systems using empirical hourly data including energy, solar irradiation and albedo, and open-source image processing methods from images of the arrays in a northern environment in the winter. Projection transformations based on reference anchor points and snowless ground truth images provide reliable masking and optical distortion correction with fixed surveillance cameras. …

Multiple Object Tracking For Marine Science, Nicholas A. Wachter

Computer Science and Software Engineering

As drone and computer vision technology has been improving, many fields of study have been quick to utilize it to improve the accuracy and ease of data collection. The combination of the two technologies is perfect for surveying large areas and identifying features of interest. Marine science utilizes these technologies for activities such as animal tracking and population counting. I am working with the Drones for Marine Science research group at Cal Poly who want to build a fleet of drones that will fly out over the ocean to identify and track various marine animals. My role will be to …

Weakly-Supervised Tumor Purity Prediction From Frozen H&E Stained Slides, Matthew Brendel, Vanesa Getseva, Majd Al Assaad, Michael Sigouros, Alexandros Sigaras, Troy Kane, Pegah Khosravi, Juan Miguel Mosquera, Olivier Elemento, Iman Hajirasouliha

Publications and Research

Background

Estimating tumor purity is especially important in the age of precision medicine. Purity estimates have been shown to be critical for correction of tumor sequencing results, and higher purity samples allow for more accurate interpretations from next-generation sequencing results. Molecular-based purity estimates using computational approaches require sequencing of tumors, which is both time-consuming and expensive.

Methods

Here we propose an approach, weakly-supervised purity (wsPurity), which can accurately quantify tumor purity within a digitally captured hematoxylin and eosin (H&E) stained histological slide, using several types of cancer from The Cancer Genome Atlas (TCGA) as a proof-of-concept.

Findings

Our model predicts …

Comparing Learned Representations Between Unpruned And Pruned Deep Convolutional Neural Networks, Parker Mitchell

Master's Theses

While deep neural networks have shown impressive performance in computer vision tasks, natural language processing, and other domains, the sizes and inference times of these models can often prevent them from being used on resource-constrained systems. Furthermore, as these networks grow larger in size and complexity, it can become even harder to understand the learned representations of the input data that these networks form through training. These issues of growing network size, increasing complexity and runtime, and ambiguity in the understanding of internal representations serve as guiding points for this work.

In this thesis, we create a neural network that …

A Unified View Of A Human Digital Twin, Michael Miller, Emily Spatz

Faculty Publications

The term human digital twin has recently been applied in many domains, including medical and manufacturing. This term extends the digital twin concept, which has been illustrated to provide enhanced system performance as it combines system models and analyses with real-time measurements for an individual system to improve system maintenance. Human digital twins have the potential to change the practice of human system integration as these systems employ real-time sensing and feedback to tightly couple measurements of human performance, behavior, and environmental influences throughout a product’s life cycle to human models to improve system design and performance. However, as this …

Leveraging Context Patterns For Medical Entity Classification, Garrett Johnston

Computer Science Senior Theses

The ability of patients to understand health-related text is important for optimal health outcomes. A system that can automatically annotate medical entities could help patients better understand health-related text. Such a system would also accelerate manual data annotation for this low-resource domain as well as assist in down- stream medical NLP tasks such as finding textual similarity, identifying conflicting medical advice, and aspect-based sentiment analysis. In this work, we investigate a state-of-the-art entity set expansion model, BootstrapNet, for the task of medical entity classification on a new dataset of medical advice text. We also propose EP SBERT, a simple model …

Symplectically Integrated Symbolic Regression Of Hamiltonian Dynamical Systems, Daniel Dipietro

Computer Science Senior Theses

Here we present Symplectically Integrated Symbolic Regression (SISR), a novel technique for learning physical governing equations from data. SISR employs a deep symbolic regression approach, using a multi-layer LSTMRNN with mutation to probabilistically sample Hamiltonian symbolic expressions. Using symplectic neural networks, we develop a model-agnostic approach for extracting meaningful physical priors from the data that can be imposed on-the-fly into the RNN output, limiting its search space. Hamiltonians generated by the RNN are optimized and assessed using a fourth-order symplectic integration scheme; prediction performance is used to train the LSTM-RNN to generate increasingly better functions via a risk-seeking policy gradients …

Single-Pass Inline Pipeline 3d Reconstruction Using Depth Camera Array, Zhexiong Shang, Zhigang Shen

Department of Construction Engineering and Management: Faculty Publications

A novel inline inspection (ILI) approach using depth cameras array (DCA) is introduced to create high-fidelity, dense 3D pipeline models. A new camera calibration method is introduced to register the color and the depth information of the cameras into a unified pipe model. By incorporating the calibration outcomes into a robust camera motion estimation approach, dense and complete 3D pipe surface reconstruction is achieved by using only the inline image data collected by a self-powered ILI rover in a single pass through a straight pipeline. The outcomes of the laboratory experiments demonstrate one-millimeter geometrical accuracy and 0.1-pixel photometric accuracy. …

Where Is The Author: The Copyright Protection For Ai-Generated Works, Chieh Huang

Maurer Theses and Dissertations

The two groups of the human-or-machine questions, whether AI-generated works are copyrightable and whether AI-generated works have human authors, are revisiting the current copyright law with the emergence of AI-generated works. These revisiting questions reveal that the current authorship requirement fails to provide a clear and operable standard on evaluating a human contributor’s intellectual labor for creative output. Such a defect of the current authorship requirement has to be fixed to respond to the technological change of artificial intelligence and the burgeoning prevalence of AI- or advanced computer program-generated works.

This dissertation’s main goal is to fix the flaw …

Multimodal Zero-Shot Hateful Meme Detection, Jiawen Zhu, Roy Ka-Wei Lee, Wen Haw Chong

Research Collection School Of Computing and Information Systems

Facebook has recently launched the hateful meme detection challenge, which garnered much attention in academic and industry research communities. Researchers have proposed multimodal deep learning classification methods to perform hateful meme detection. While the proposed methods have yielded promising results, these classification methods are mostly supervised and heavily rely on labeled data that are not always available in the real-world setting. Therefore, this paper explores and aims to perform hateful meme detection in a zero-shot setting. Working towards this goal, we propose Target-Aware Multimodal Enhancement (TAME), which is a novel deep generative framework that can improve existing hateful meme classification …

High-Resolution Face Swapping Via Latent Semantics Disentanglement, Yangyang Xu, Bailin Deng, Junle Wang, Yanqing Jing, Jia Pan, Shengfeng He

Research Collection School Of Computing and Information Systems

We present a novel high-resolution face swapping method using the inherent prior knowledge of a pre-trained GAN model. Although previous research can leverage generative priors to produce high-resolution results, their quality can suffer from the entangled semantics of the latent space. We explicitly disentangle the latent semantics by utilizing the progressive nature of the generator, deriving structure at-tributes from the shallow layers and appearance attributes from the deeper ones. Identity and pose information within the structure attributes are further separated by introducing a landmark-driven structure transfer latent direction. The disentangled latent code produces rich generative features that incorporate feature blending …

Deep Learning For Anomaly Detection, Guansong Pang, Charu Aggarwal, Chunhua Shen, Nicu Sebe

Research Collection School Of Computing and Information Systems

A nomaly detection aims at identifying data points which are rare or significantly different from the majority of data points. Many techniques are explored to build highly efficient and effective anomaly detection systems, but they are confronted with many difficulties when dealing with complex data, such as failing to capture intricate feature interactions or extract good feature representations. Deep-learning techniques have shown very promising performance in tackling different types of complex data in a broad range of tasks/problems, including anomaly detection. To address this new trend, we organized this Special Issue on Deep Learning for Anomaly Detection to cover the …

Mems Ultrasonic Transducers For Safe, Low-Power And Portable Eye-Blinking Monitoring, Sheng Sun, Jianyuan Wang, Menglun Zhang, Yuan Ning, Dong Ma, Yi Yuan, Pengfei Niu, Zhicong Rong, Zhuochen Wang, Wei Pang

Research Collection School Of Computing and Information Systems

Eye blinking is closely related to human physiology and psychology. It is an effective method of communication among people and can be used in human–machine interactions. Existing blink monitoring methods include video-oculography, electro-oculograms and infrared oculography. However, these methods suffer from uncomfortable use, safety risks, limited reliability in strong light or dark environments, and infringed informational security. In this paper, we propose an ultrasound-based portable approach for eye-blinking activity monitoring. Low-power pulse-echo ultrasound featuring biosafety is transmitted and received by microelectromechanical system (MEMS) ultrasonic transducers seamlessly integrated on glasses. The size, weight and power consumption of the transducers are 2.5 …

Imagining New Futures Beyond Predictive Systems In Child Welfare: A Qualitative Study With Impacted Stakeholders, Logan Stapleton, Min Hun Lee, Diana Qing, Marya Wright, Alexandra Chouldechova, Ken Holstein, Zhiwei Steven Wu, Haiyi Zhu

Research Collection School Of Computing and Information Systems

Child welfare agencies across the United States are turning to datadriven predictive technologies (commonly called predictive analytics) which use government administrative data to assist workers’ decision-making. While some prior work has explored impacted stakeholders’ concerns with current uses of data-driven predictive risk models (PRMs), less work has asked stakeholders whether such tools ought to be used in the first place. In this work, we conducted a set of seven design workshops with 35 stakeholders who have been impacted by the child welfare system or who work in it to understand their beliefs and concerns around PRMs, and to engage them …

Cross-Lingual Adaptation For Recipe Retrieval With Mixup, Bin Zhu, Chong-Wah Ngo, Jingjing Chen, Wing-Kwong Chan

Research Collection School Of Computing and Information Systems

Cross-modal recipe retrieval has attracted research attention in recent years, thanks to the availability of large-scale paired data for training. Nevertheless, obtaining adequate recipe-image pairs covering the majority of cuisines for supervised learning is difficult if not impossible. By transferring knowledge learnt from a data-rich cuisine to a data-scarce cuisine, domain adaptation sheds light on this practical problem. Nevertheless, existing works assume recipes in source and target domains are mostly originated from the same cuisine and written in the same language. This paper studies unsupervised domain adaptation for image-to-recipe retrieval, where recipes in source and target domains are in different …

Reinforcement Learning-Based Interactive Video Search, Zhixin Ma, Jiaxin Wu, Zhijian Hou, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Despite the rapid progress in text-to-video search due to the advancement of cross-modal representation learning, the existing techniques still fall short in helping users to rapidly identify the search targets. Particularly, in the situation that a system suggests a long list of similar candidates, the user needs to painstakingly inspect every search result. The experience is frustrated with repeated watching of similar clips, and more frustratingly, the search targets may be overlooked due to mental tiredness. This paper explores reinforcement learning-based (RL) searching to relieve the user from the burden of brute force inspection. Specifically, the system maintains a graph …

Group Contextualization For Video Recognition, Yanbin Hao, Hao Zhang, Chong-Wah Ngo, Xiangnan He

Research Collection School Of Computing and Information Systems

Learning discriminative representation from the complex spatio-temporal dynamic space is essential for video recognition. On top of those stylized spatio-temporal computational units, further refining the learnt feature with axial contexts is demonstrated to be promising in achieving this goal. However, previous works generally focus on utilizing a single kind of contexts to calibrate entire feature channels and could hardly apply to deal with diverse video activities. The problem can be tackled by using pair-wise spatio-temporal attentions to recompute feature response with cross-axis contexts at the expense of heavy computations. In this paper, we propose an efficient feature refinement method that …

Mlp-3d: A Mlp-Like 3d Architecture With Grouped Time Mixing, Zhaofan Qiu, Ting Yao, Chong-Wah Ngo, Tao Mei

Research Collection School Of Computing and Information Systems

Convolutional Neural Networks (CNNs) have been re-garded as the go-to models for visual recognition. More re-cently, convolution-free networks, based on multi-head self-attention (MSA) or multi-layer perceptrons (MLPs), become more and more popular. Nevertheless, it is not trivial when utilizing these newly-minted networks for video recognition due to the large variations and complexities in video data. In this paper, we present MLP-3D networks, a novel MLP-like 3D architecture for video recognition. Specifically, the architecture consists of MLP-3D blocks, where each block contains one MLP applied across tokens (i.e., token-mixing MLP) and one MLP applied independently to each token (i.e., channel MLP). …

Catching Both Gray And Black Swans: Open-Set Supervised Anomaly Detection, Choubo Ding, Guansong Pang, Chunhua Shen

Research Collection School Of Computing and Information Systems

Despite most existing anomaly detection studies assume the availability of normal training samples only, a few labeled anomaly examples are often available in many real-world applications, such as defect samples identified during random quality inspection, lesion images confirmed by radiologists in daily medical screening, etc. These anomaly examples provide valuable knowledge about the application-specific abnormality, enabling significantly improved detection of similar anomalies in some recent models. However, those anomalies seen during training often do not illustrate every possible class of anomaly, rendering these models ineffective in generalizing to unseen anomaly classes. This paper tackles open-set supervised anomaly detection, in which …

Generative Flows With Invertible Attentions, Rhea Sanjay Sukthanker, Zhiwu Huang, Suryansh Kumar, Radu Timofte, Luc Van Gool

Research Collection School Of Computing and Information Systems

Flow-based generative models have shown an excellent ability to explicitly learn the probability density function of data via a sequence of invertible transformations. Yet, learning attentions in generative flows remains understudied, while it has made breakthroughs in other domains. To fill the gap, this paper introduces two types of invertible attention mechanisms, i.e., map-based and transformer-based attentions, for both unconditional and conditional generative flows. The key idea is to exploit a masked scheme of these two attentions to learn long-range data dependencies in the context of generative flows. The masked scheme allows for invertible attention modules with tractable Jacobian determinants, …

Officers: Operational Framework For Intelligent Crime-And-Emergency Response Scheduling, Jonathan David Chase, Siong Thye Goh, Tran Phong, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

In the quest to achieve better response times in dense urban environments, law enforcement agencies are seeking AI-driven planning systems to inform their patrol strategies. In this paper, we present a framework, OFFICERS, for deployment planning that learns from historical data to generate deployment schedules on a daily basis. We accurately predict incidents using ST-ResNet, a deep learning technique that captures wide-ranging spatio-temporal dependencies, and solve a large-scale optimization problem to schedule deployment, significantly improving its scalability through a simulated annealing solver. Methodologically, our approach outperforms our previous works where prediction was done using Generative Adversarial Networks, and optimization was …

Reinforcement Learning Approach To Solve Dynamic Bi-Objective Police Patrol Dispatching And Rescheduling Problem, Waldy Joe, Hoong Chuin Lau, Jonathan Pan

Research Collection School Of Computing and Information Systems

Police patrol aims to fulfill two main objectives namely to project presence and to respond to incidents in a timely manner. Incidents happen dynamically and can disrupt the initially-planned patrol schedules. The key decisions to be made will be which patrol agent to be dispatched to respond to an incident and subsequently how to adapt the patrol schedules in response to such dynamically-occurring incidents whilst still fulfilling both objectives; which sometimes can be conflicting. In this paper, we define this real-world problem as a Dynamic Bi-Objective Police Patrol Dispatching and Rescheduling Problem and propose a solution approach that combines Deep …

Do-Gan: A Double Oracle Framework For Generative Adversarial Networks, Aye Phyu Phye Aung, Xinrun Wang, Runsheng Yu, Bo An, Senthilnath Jayavelu, Xiaoli Li

Research Collection School Of Computing and Information Systems

In this paper, we propose a new approach to train Gen-erative Adversarial Networks (GANs) where we deploy a double-oracle framework using the generator and discrim-inator oracles. GAN is essentially a two-player zero-sum game between the generator and the discriminator. Training GANs is challenging as a pure Nash equilibrium may not exist and even finding the mixed Nash equilibrium is difficult as GANs have a large-scale strategy space. In DO-GAN, we extend the double oracle framework to GANs. We first generalize the players' strategies as the trained models of generator and discriminator from the best response or-acles. We then compute the …

Effect Of Monetary Policy Rate On Market Interest Rates In Nigeria: A Threshold And Nardl Approach, Oluwafemi E. Awopegba, Joseph O. Afolabi, Lydia T. Adeoye, Godwin O. Akpokodje

CBN Journal of Applied Statistics (JAS)

This study examines the effect of monetary policy rate (MPR) on market interest rates in Nigeria. For parsimony, we develop two indexes called the short-term interest rate (SINT) and Lending interest rate (LINT) to represent deposit and lending rates respectively. The nonlinear autoregressive distributed lag (NARDL) and threshold regression models are adopted. The study uses monthly data from 2002:M1 to 2019:M12. The results of the threshold regression model indicate that the degree of the effect of MPR on SINT and LINT above the estimated threshold of 11 and 13 percent respectively is greater and significant than if MPR were to …

Social Dimension Of Inclusive Growth In Ecowas: Implication For Poverty Reduction, Toriola K. Anu, Goerge O. Emmanuel, Ajayi O. Felix

CBN Journal of Applied Statistics (JAS)

This study investigates the implication of the social dimension of inclusive growth on poverty reduction in Economic Community of West African States (ECOWAS) countries. It specifically examines how social indices of inclusive growth comprising of income inequality, education, and health outcomes affect poverty reduction. The study uses a panel dataset of the six (6) lower-middle income countries in ECOWAS which was analysed via panel Difference Generalised Method of Moment (D-GMM). The results show that GDP per capita exerts significant negative effect on poverty while inequality, education and health outcomes do not show significant effect on poverty. Although, the estimates of …

Effect Of Fdi Inflows On Employment Generation In Selected Ecowas Countries: Heterogeneous Panel Analysis, Timothy A. Aderemi, Olawunmi Omitogun, Bukonla G. Osisanwo

CBN Journal of Applied Statistics (JAS)

The aim of this study is to examine the effect of FDI on employment in ECOWAS sub region between 1990 and 2019. The study utilizes a panel autoregressive distributed lag model to analyse the short run and long run relationship between FDI and employment across ECOWAS sub region. In the short run, the impact of FDI on employment is negative and statistically not significant. Meanwhile, in the long run FDI has a positive and statistically significant impact on employment rate. This implies that FDI has the capacity to generate employment in countries in ECOWAS sub region. Therefore, this study recommends …

Impact Of Covid-19 Pandemic On The Nigeria Stock Market: A Sectoral Stock Prices Analysis, Peter A. Adekunle, Yakubu A. Bello, Udochukwu G. Nwachukwu

CBN Journal of Applied Statistics (JAS)

This study examines the impact of the COVID-19 pandemic on sectoral stock prices in Nigeria stock market using daily data covering from February 28, 2020 to June 26, 2020. Applying the autoregressive distributed lag (ARDL) bounds test, the study finds that COVID-19 pandemic had adverse impact on the stock market indices in the short run. Furthermore, the study documents negative response of sectoral stock prices to the pandemic while the stock prices of the banking sub-sector are the worst hit. Compared to the consumer goods, and industrial subsector indices, the speed of adjustment to long run equilibrium is faster for …

Wildfire Risk Assessment Using Convolutional Neural Networks And Modis Climate Data, Sean F. Nesbit

Master's Theses

Wildfires burn millions of acres of land each year leading to the destruction of homes and wildland ecosystems while costing governments billions in funding. As climate change intensifies drought volatility across the Western United States, wildfires are likely to become increasingly severe. Wildfire risk assessment and hazard maps are currently employed by fire services, but can often be outdated. This paper introduces an image-based dataset using climate and wildfire data from NASA’s Moderate Resolution Imaging Spectroradiometer (MODIS). The dataset consists of 32 climate and topographical layers captured across 0.1 deg by 0.1 deg tiled regions in California and Nevada between …