Physical Sciences and Mathematics Commons^™

Combining Empirical And Physics-Based Models For Solar Wind Prediction, Rob Johnson, Soukaina Filali Boubrahimi, Omar Bahri, Shah Muhammad Hamdi

Computer Science Faculty and Staff Publications

Solar wind modeling is classified into two main types: empirical models and physics-based models, each designed to forecast solar wind properties in various regions of the heliosphere. Empirical models, which are cost-effective, have demonstrated significant accuracy in predicting solar wind at the L1 Lagrange point. On the other hand, physics-based models rely on magnetohydrodynamics (MHD) principles and demand more computational resources. In this research paper, we build upon our recent novel approach that merges empirical and physics-based models. Our recent proposal involves the creation of a new physics-informed neural network that leverages time series data from solar wind predictors to …

Numerical Simulations For Fractional Differential Equations Of Higher Order And A Wright-Type Transformation, Mariana Nacianceno, Tamer Oraby, Hansapani Rodrigo, Y. Sepulveda, Josef A. Sifuentes, Erwin Suazo, T. Stuck, J. Williams

School of Mathematical and Statistical Sciences Faculty Publications and Presentations

In this work, a new relationship is established between the solutions of higher fractional differential equations and a Wright-type transformation. Solutions could be interpreted as expected values of functions in a random time process. As applications, we solve the fractional beam equation, fractional electric circuits with special functions as external sources, and derive d’Alembert’s formula for the fractional wave equation. Due to this relationship, we present two methods for simulating solutions of fractional differential equations. The two approaches use the interpretation of the Caputo derivative of a function as a Wright-type transformation of the higher derivative of the function. In …

Continual Online Learning-Based Optimal Tracking Control Of Nonlinear Strict-Feedback Systems: Application To Unmanned Aerial Vehicles, Irfan Ganie, Sarangapani Jagannathan

Electrical and Computer Engineering Faculty Research & Creative Works

A novel optimal trajectory tracking scheme is introduced for nonlinear continuous-time systems in strict feedback form with uncertain dynamics by using neural networks (NNs). The method employs an actor-critic-based NN back-stepping technique for minimizing a discounted value function along with an identifier to approximate unknown system dynamics that are expressed in augmented form. Novel online weight update laws for the actor and critic NNs are derived by using both the NN identifier and Hamilton-Jacobi-Bellman residual error. A new continual lifelong learning technique utilizing the Fisher Information Matrix via Hamilton-Jacobi-Bellman residual error is introduced to obtain the significance of weights in …

Conditional Neural Heuristic For Multiobjective Vehicle Routing Problems, Mingfeng Fan, Yaoxin Wu, Zhiguang Cao, Wen Song, Guillaume Sartoretti, Huan Liu, Guohua Wu

Research Collection School Of Computing and Information Systems

Existing neural heuristics for multiobjective vehicle routing problems (MOVRPs) are primarily conditioned on instance context, which failed to appropriately exploit preference and problem size, thus holding back the performance. To thoroughly unleash the potential, we propose a novel conditional neural heuristic (CNH) that fully leverages the instance context, preference, and size with an encoder–decoder structured policy network. Particularly, in our CNH, we design a dual-attention-based encoder to relate preferences and instance contexts, so as to better capture their joint effect on approximating the exact Pareto front (PF). We also design a size-aware decoder based on the sinusoidal encoding to explicitly …

Predicting Forage Provision Of Grasslands Across Climate Zones By Hyperspectral Measurements, F. A. Männer, J. Muro, J. Ferner, S. Schmidtlein, A. Linstädter

IGC Proceedings (1997-2023)

The potential of grasslands’ fodder production is a crucial management measure, while its quantification is still laborious and costly. Remote sensing technologies, such as hyperspectral field measurements, enable fast and non-destructive estimation. However, such methods are still limited in transferability to other locations or climatic conditions. With this study, we aim to predict forage nutritive value, quantity, and energy yield from hyperspectral canopy reflections of grasslands across three climate zones. We took hyperspectral measurements with a field spectrometer from grassland canopies in temperate, tropical and semi-arid grasslands, and analyzed corresponding biomass samples for their quantity (BM), metabolizable energy content (ME) …

Open System Neural Networks, Bradley Hatch

Theses and Dissertations

Recent advances in self-supervised learning have made it possible to reuse information-rich models that have been generally pre-trained on massive amounts of data for other downstream tasks. But the pre-training process can be drastically different from the fine-tuning training process, which can lead to inefficient learning. We address this disconnect in training dynamics by structuring the learning process like an open system in thermodynamics. Open systems can achieve a steady state when low-entropy inputs are converted to high-entropy outputs. We modify the the model and the learning process to mimic this behavior, and attend more to elements of the input …

Automatic Classification Of Activities In Classroom Videos, Jonathan K. Foster, Matthew Korban, Peter Youngs, Ginger S. Watson, Scott T. Acton

VMASC Publications

Classroom videos are a common source of data for educational researchers studying classroom interactions as well as a resource for teacher education and professional development. Over the last several decades emerging technologies have been applied to classroom videos to record, transcribe, and analyze classroom interactions. With the rise of machine learning, we report on the development and validation of neural networks to classify instructional activities using video signals, without analyzing speech or audio features, from a large corpus of nearly 250 h of classroom videos from elementary mathematics and English language arts instruction. Results indicated that the neural networks performed …

On Generative Models And Joint Architectures For Document-Level Relation Extraction, Aviv Brokman

Theses and Dissertations--Statistics

Biomedical text is being generated at a high rate in scientific literature publications and electronic health records. Within these documents lies a wealth of potentially useful information in biomedicine. Relation extraction (RE), the process of automating the identification of structured relationships between entities within text, represents a highly sought-after goal in biomedical informatics, offering the potential to unlock deeper insights and connections from this vast corpus of data. In this dissertation, we tackle this problem with a variety of approaches.

We review the recent history of the field of document-level RE. Several themes emerge. First, graph neural networks dominate the …

Distxplore: Distribution-Guided Testing For Evaluating And Enhancing Deep Learning Systems, Longtian Wang, Xiaofei Xie, Xiaoning Du, Meng Tian, Qing Guo, Zheng Yang, Chao Shen

Research Collection School Of Computing and Information Systems

Deep learning (DL) models are trained on sampled data, where the distribution of training data differs from that of real-world data (i.e., the distribution shift), which reduces the model's robustness. Various testing techniques have been proposed, including distribution-unaware and distribution-aware methods. However, distribution-unaware testing lacks effectiveness by not explicitly considering the distribution of test cases and may generate redundant errors (within same distribution). Distribution-aware testing techniques primarily focus on generating test cases that follow the training distribution, missing out-of-distribution data that may also be valid and should be considered in the testing process. In this paper, we propose a novel …

Uavs And Deep Neural Networks: An Alternative Approach To Monitoring Waterfowl At The Site Level, Zachary J. Loken

LSU Master's Theses

Understanding how waterfowl respond to habitat restoration and management activities is crucial for evaluating and refining conservation delivery programs. However, site-specific waterfowl monitoring is challenging, especially in heavily forested systems such as the Mississippi Alluvial Valley (MAV)—a primary wintering region for ducks in North America. I hypothesized that using uncrewed aerial vehicles (UAVs) coupled with deep learning-based methods for object detection would provide an efficient and effective means for surveying non-breeding waterfowl on difficult-to-access restored wetland sites. Accordingly, during the winters of 2021 and 2022, I surveyed wetland restoration easements in the MAV using a UAV equipped with a dual …

Faire: Repairing Fairness Of Neural Networks Via Neuron Condition Synthesis, Tianlin Li, Xiaofei Xie, Jian Wang, Qing Guo, Aishan Liu, Lei Ma, Yang Liu

Research Collection School Of Computing and Information Systems

Deep Neural Networks (DNNs) have achieved tremendous success in many applications, while it has been demonstrated that DNNs can exhibit some undesirable behaviors on concerns such as robustness, privacy, and other trustworthiness issues. Among them, fairness (i.e., non-discrimination) is one important property, especially when they are applied to some sensitive applications (e.g., finance and employment). However, DNNs easily learn spurious correlations between protected attributes (e.g., age, gender, race) and the classification task and develop discriminatory behaviors if the training data is imbalanced. Such discriminatory decisions in sensitive applications would introduce severe social impacts. To expose potential discrimination problems in DNNs …

Investigating Continual Learning Strategies In Neural Networks, Christopher Tam, Luiz Fernando Capretz

Electrical and Computer Engineering Publications

This paper explores the role of continual learning strategies when neural networks are confronted with learning tasks sequentially. We analyze the stability-plasticity dilemma with three factors in mind: the type of network architecture used, the continual learning scenario defined and the continual learning strategy implemented. Our results show that complementary learning systems and neural volume significantly contribute towards memory retrieval and consolidation in neural networks. Finally, we demonstrate how regularization strategies such as elastic weight consolidation are more well-suited for larger neural networks whereas rehearsal strategies such as gradient episodic memory are better suited for smaller neural networks.

A System For The Detection Of Adversarial Attacks In Computer Vision Via Performance Metrics, Sarah Reynolds

Doctoral Dissertations and Master's Theses

Adversarial attacks, or attacks committed by an adversary to hijack a system, are prevalent in the deep learning tasks of computer vision and are one of the greatest threats to these models' safe and accurate use. These attacks force the trained model to misclassify an image, using pixel-level changes undetectable to the human eye. Various defenses against these attacks exist and are detailed in this work. The work of previous researchers has established that when adversarial attacks occur, different node patterns in a Deep Neural Network (DNN) are activated within the model. Additionally, it is known that CPU and GPU …

Deep Reinforcement Learning With Explicit Context Representation, Francisco Munguia-Galeano, Ah-Hwee Tan, Ze Ji

Research Collection School Of Computing and Information Systems

Though reinforcement learning (RL) has shown an outstanding capability for solving complex computational problems, most RL algorithms lack an explicit method that would allow learning from contextual information. On the other hand, humans often use context to identify patterns and relations among elements in the environment, along with how to avoid making wrong actions. However, what may seem like an obviously wrong decision from a human perspective could take hundreds of steps for an RL agent to learn to avoid. This article proposes a framework for discrete environments called Iota explicit context representation (IECR). The framework involves representing each state …

Effects Of Weight Initialization Methods On Ffn's, Ida K. Karem

The Cardinal Edge

Weight initialization is the method of determining starting values of weights in a neural network. The way this method is done can have massive effects on the network[2, 3, 6, 9] and can halt training if not handled properly. On the other hand, if initialization is chosen tactfully it can improve training and accuracy greatly. The initialization method usually called Normalized Xavier will be referred to as Nox in this paper to avoid confusion with the Xavier initialization method. This study analyzes five methods of weight initialization(Nox, He, Xavier, Plutonian, and Self-Root), two of them …

Physics-Guided Deep Learning For Solar Wind Modeling At L1 Point, Robert M. Johnson

All Graduate Theses and Dissertations, Spring 1920 to Summer 2023

Neural networks are adept at finding patterns that are too long and too small for humans to find in data. Usually, this power is used to generate predictions with greater accuracy than most alternative models. However, we can also use this power to understand more about the data we train these networks on. We do this by changing the data that the networks train on and the data they are tested on. This allows us to both control the maximum length of a pattern and to compare data between different groups, in our case, different solar cycles. This thesis is …

Lidar Segmentation-Based Adversarial Attacks On Autonomous Vehicles, Blake Johnson

Scholarly Horizons: University of Minnesota, Morris Undergraduate Journal

Autonomous vehicles utilizing LiDAR-based 3D perception systems are susceptible to adversarial attacks. This paper focuses on a specific attack scenario that relies on the creation of adversarial point clusters with the intention of fooling the segmentation model utilized by LiDAR into misclassifying point cloud data. This can be translated into the real world with the placement of objects (such as road signs or cardboard) at these adversarial point cluster locations. These locations are generated through an optimization algorithm performed on said adversarial point clusters that are introduced by the attacker.

Learning The Game: Implementations Of Convolutional Networks In Automated Strategy Identification, Cameron Klig

Master's Theses

Games can be used to represent a wide variety of real world problems, giving rise to many applications of game theory. Various computational methods have been proposed for identifying game strategies, including optimized tree search algorithms, game-specific heuristics, and artificial intelligence. In the last decade, systems like AlphaGo and AlphaZero have significantly exceeded the performance of the best human players in Chess, Go, and other games. The most effective game engines to date employ convolutional neural networks (CNNs) to evaluate game boards, extract features, and predict the optimal next move. These engines are trained on billions of simulated games, wherein …

Integration Of Neural Network And Distance Relay To Improve The Fault Localization On Transmission Lines, Linh Tran

Turkish Journal of Electrical Engineering and Computer Sciences

Power transmission lines are integral and very important components of power systems. Because of the length of these lines and the complexity of the power grids, the lines may encounter various incidents such as lightning strike, shortage, and breakage. When an incident or a fault occurs, a fast process of identification, localization, and isolation of the fault is desired. An accurate fault localization would have a great impact in reducing the restoration time of the system. One of the most popular solutions for fault detection and localization is the distance relays using the impedance-based algorithms. However, these relays are still …

Loss Scaling And Step Size In Deep Learning Optimizatio, Nora Alosily

Dissertations

Deep learning training consumes ever-increasing time and resources, and that is
due to the complexity of the model, the number of updates taken to reach good
results, and both the amount and dimensionality of the data. In this dissertation,
we will focus on making the process of training more efficient by focusing on the
step size to reduce the number of computations for parameters in each update.
We achieved our objective in two new ways: we use loss scaling as a proxy for
the learning rate, and we use learnable layer-wise optimizers. Although our work
is perhaps not the first …

Convolutional Neural Networks Analysis Reveals Three Possible Sources Of Bronze Age Writings Between Greece And India, Shruti Daggumati, Peter Z. Revesz

School of Computing: Faculty Publications

This paper analyzes the relationships among eight ancient scripts from between Greece and India. We used convolutional neural networks combined with support vector machines to give a numerical rating of the similarity between pairs of signs (one sign from each of two different scripts). Two scripts that had a one-to-one matching of their signs were determined to be related. The result of the analysis is the finding of the following three groups, which are listed in chronological order: (1) Sumerian pictograms, the Indus Valley script, and the proto-Elamite script; (2) Cretan hieroglyphs and Linear B; and (3) the Phoenician, Greek, …

Applications Of Generative Adversarial Networks In Single Image Datasets, Dylan E. Cramer

Scholarly Horizons: University of Minnesota, Morris Undergraduate Journal

One of the main difficulties faced in most generative machine learning models is how much data is required to train it, especially when collecting a large dataset is not feasible. Recently there have been breakthroughs in tackling this issue in SinGAN, with its researchers being able to train a Generative Adversarial Network (GAN) on just a single image with a model that can perform many novel tasks, such as image harmonization. ConSinGAN is a model that builds upon this work by concurrently training several stages in a sequential multi-stage manner while retaining the ability to perform those novel tasks.

Learning And Understanding User Interface Semantics From Heterogeneous Networks With Multimodal And Positional Attributes, Meng Kiat Gary Ang, Ee-Peng Lim

Research Collection School Of Computing and Information Systems

User interfaces (UI) of desktop, web, and mobile applications involve a hierarchy of objects (e.g., applications, screens, view class, and other types of design objects) with multimodal (e.g., textual and visual) and positional (e.g., spatial location, sequence order, and hierarchy level) attributes. We can therefore represent a set of application UIs as a heterogeneous network with multimodal and positional attributes. Such a network not only represents how users understand the visual layout of UIs but also influences how users would interact with applications through these UIs. To model the UI semantics well for different UI annotation, search, and evaluation tasks, …

Effectiveness Of A Timing Side Channel For Deriving Neural Network Depth, Matthew P. Weeks

Theses and Dissertations

From facial recognition on cell phones to vehicle traffic modeling for city planning, integrating ML models can be an expensive investment in resources. Protecting that investment is difficult, as information about the model and how it was built can be leaked through multiple channels, such as timing and memory access. In this thesis, one method of extracting data through a timing side-channel is examined across multiple hardware and software configurations to determine its reliability for general use. While attempting to determine the layer count of a target model solely from its inference time, the research determined that it is not …

Emotion Classification Of Indonesian Tweets Using Bidirectional Lstm, Aaron K. Glenn, Phillip M. Lacasse, Bruce A. Cox

Faculty Publications

Emotion classification can be a powerful tool to derive narratives from social media data. Traditional machine learning models that perform emotion classification on Indonesian Twitter data exist but rely on closed-source features. Recurrent neural networks can meet or exceed the performance of state-of-the-art traditional machine learning techniques using exclusively open-source data and models. Specifically, these results show that recurrent neural network variants can produce more than an 8% gain in accuracy in comparison with logistic regression and SVM techniques and a 15% gain over random forest when using FastText embeddings. This research found a statistical significance in the performance of …

Artificial Intelligence-Enabled Exploratory Cyber-Physical Safety Analyzer Framework For Civilian Urban Air Mobility, Md. Shirajum Munir, Sumit Howlader Dipro, Kamrul Hasan, Tariqul Islam, Sachin Shetty

VMASC Publications

Urban air mobility (UAM) has become a potential candidate for civilization for serving smart citizens, such as through delivery, surveillance, and air taxis. However, safety concerns have grown since commercial UAM uses a publicly available communication infrastructure that enhances the risk of jamming and spoofing attacks to steal or crash crafts in UAM. To protect commercial UAM from cyberattacks and theft, this work proposes an artificial intelligence (AI)-enabled exploratory cyber-physical safety analyzer framework. The proposed framework devises supervised learning-based AI schemes such as decision tree, random forests, logistic regression, K-nearest neighbors (KNN), and long short-term memory (LSTM) for predicting and …

Inter-Rater Agreement For The Annotation Of Neurologic Signs And Symptoms In Electronic Health Records, Chelsea Oommen, Quentin Howlett-Prieto, Michael D. Carrithers, Daniel B. Hier

Chemistry Faculty Research & Creative Works

The extraction of patient signs and symptoms recorded as free text in electronic health records is critical for precision medicine. Once extracted, signs and symptoms can be made computable by mapping to signs and symptoms in an ontology. Extracting signs and symptoms from free text is tedious and time-consuming. Prior studies have suggested that inter-rater agreement for clinical concept extraction is low. We have examined inter-rater agreement for annotating neurologic concepts in clinical notes from electronic health records. After training on the annotation process, the annotation tool, and the supporting neuro-ontology, three raters annotated 15 clinical notes in three rounds. …

Continual Learning-Based Optimal Output Tracking Of Nonlinear Discrete-Time Systems With Constraints: Application To Safe Cargo Transfer, Behzad Farzanegan, S. (Sarangapani) Jagannathan

Electrical and Computer Engineering Faculty Research & Creative Works

This Paper Addresses a Novel Lifelong Learning (LL)-Based Optimal Output Tracking Control of Uncertain Non-Linear Affine Discrete-Time Systems (DT) with State Constraints. First, to Deal with Optimal Tracking and Reduce the Steady State Error, a Novel Augmented System, Including Tracking Error and its Integral Value and Desired Trajectory, is Proposed. to Guarantee Safety, an Asymmetric Barrier Function (BF) is Incorporated into the Utility Function to Keep the Tracking Error in a Safe Region. Then, an Adaptive Neural Network (NN) Observer is Employed to Estimate the State Vector and the Control Input Matrix of the Uncertain Nonlinear System. Next, an NN-Based …

Adversarial Training Of Deep Neural Networks, Anabetsy Termini

CCE Theses and Dissertations

Deep neural networks used for image classification are highly susceptible to adversarial attacks. The de facto method to increase adversarial robustness is to train neural networks with a mixture of adversarial images and unperturbed images. However, this method leads to robust overfitting, where the network primarily learns to recognize one specific type of attack used to generate the images while remaining vulnerable to others after training. In this dissertation, we performed a rigorous study to understand whether combinations of state of the art data augmentation methods with Stochastic Weight Averaging improve adversarial robustness and diminish adversarial overfitting across a wide …

Machine Learning Prediction Of Dod Personal Property Shipment Costs, Tiffany Tucker [*], Torrey J. Wagner, Paul Auclair, Brent T. Langhals

Faculty Publications

U.S. Department of Defense (DoD) personal property moves account for 15% of all domestic and international moves - accurate prediction of their cost could draw attention to outlier shipments and improve budget planning. In this work 136,140 shipments between 13 personal property shipment hubs from April 2022 through March 2023 with a total cost of $1.6B were analyzed. Shipment cost was predicted using recursive feature elimination on linear regression and XGBoost algorithms, as well as through neural network hyperparameter sweeps. Modeling was repeated after removing 28 features related to shipment hub location and branch of service to examine their influence …