Physical Sciences and Mathematics Commons^™

Numerical Simulation Of The Korteweg–De Vries Equation With Machine Learning, Kristina O. F. Williams *, Benjamin F. Akers

Faculty Publications

A machine learning procedure is proposed to create numerical schemes for solutions of nonlinear wave equations on coarse grids. This method trains stencil weights of a discretization of the equation, with the truncation error of the scheme as the objective function for training. The method uses centered finite differences to initialize the optimization routine and a second-order implicit-explicit time solver as a framework. Symmetry conditions are enforced on the learned operator to ensure a stable method. The procedure is applied to the Korteweg–de Vries equation. It is observed to be more accurate than finite difference or spectral methods on coarse …

Emotion Classification Of Indonesian Tweets Using Bidirectional Lstm, Aaron K. Glenn, Phillip M. Lacasse, Bruce A. Cox

Faculty Publications

Emotion classification can be a powerful tool to derive narratives from social media data. Traditional machine learning models that perform emotion classification on Indonesian Twitter data exist but rely on closed-source features. Recurrent neural networks can meet or exceed the performance of state-of-the-art traditional machine learning techniques using exclusively open-source data and models. Specifically, these results show that recurrent neural network variants can produce more than an 8% gain in accuracy in comparison with logistic regression and SVM techniques and a 15% gain over random forest when using FastText embeddings. This research found a statistical significance in the performance of …

Data Augmentation For Neutron Spectrum Unfolding With Neural Networks, James Mcgreivy, Juan J. Manfredi, Daniel Siefman

Faculty Publications

Neural networks require a large quantity of training spectra and detector responses in order to learn to solve the inverse problem of neutron spectrum unfolding. In addition, due to the under-determined nature of unfolding, non-physical spectra which would not be encountered in usage should not be included in the training set. While physically realistic training spectra are commonly determined experimentally or generated through Monte Carlo simulation, this can become prohibitively expensive when considering the quantity of spectra needed to effectively train an unfolding network. In this paper, we present three algorithms for the generation of large quantities of realistic and …

Machine Learning Prediction Of Dod Personal Property Shipment Costs, Tiffany Tucker [*], Torrey J. Wagner, Paul Auclair, Brent T. Langhals

Faculty Publications

U.S. Department of Defense (DoD) personal property moves account for 15% of all domestic and international moves - accurate prediction of their cost could draw attention to outlier shipments and improve budget planning. In this work 136,140 shipments between 13 personal property shipment hubs from April 2022 through March 2023 with a total cost of $1.6B were analyzed. Shipment cost was predicted using recursive feature elimination on linear regression and XGBoost algorithms, as well as through neural network hyperparameter sweeps. Modeling was repeated after removing 28 features related to shipment hub location and branch of service to examine their influence …

Generating Realistic Cyber Data For Training And Evaluating Machine Learning Classifiers For Network Intrusion Detection Systems, Marc W. Chalé, Nathaniel D. Bastian

Faculty Publications

No abstract provided.

Application Of Machine Learning To Predict The Performance Of An Emipg Reactor Using Data From Numerical Simulations, Owen Sedej, Eric G. Mbonimpa, Trevor Sleight, Jeremy Slagley

Faculty Publications

Microwave-driven plasma gasification technology has the potential to produce clean energy from municipal and industrial solid wastes. It can generate temperatures above 2000 K (as high as 30,000 K) in a reactor, leading to complete combustion and reduction of toxic byproducts. Characterizing complex processes inside such a system is however challenging. In previous studies, simulations using computational fluid dynamics (CFD) produced reproducible results, but the simulations are tedious and involve assumptions. In this study, we propose machine-learning models that can be used in tandem with CFD, to accelerate high-fidelity fluid simulation, improve turbulence modeling, and enhance reduced-order models. A two-dimensional …

Development Of Advanced Machine Learning Models For Analysis Of Plutonium Surrogate Optical Emission Spectra, Ashwin P. Rao, Phillip R. Jenkins, John D. Auxier Ii, Michael B. Shattan, Anil Patnaik

Faculty Publications

This work investigates and applies machine learning paradigms seldom seen in analytical spectroscopy for quantification of gallium in cerium matrices via processing of laser-plasma spectra. Ensemble regressions, support vector machine regressions, Gaussian kernel regressions, and artificial neural network techniques are trained and tested on cerium-gallium pellet spectra. A thorough hyperparameter optimization experiment is conducted initially to determine the best design features for each model. The optimized models are evaluated for sensitivity and precision using the limit of detection (LoD) and root mean-squared error of prediction (RMSEP) metrics, respectively. Gaussian kernel regression yields the superlative predictive model with an RMSEP of …

Machine Learning Land Cover And Land Use Classification Of 4-Band Satellite Imagery, Lorelei Turner [*], Torrey J. Wagner, Paul Auclair, Brent T. Langhals

Faculty Publications

Land-cover and land-use classification generates categories of terrestrial features, such as water or trees, which can be used to track how land is used. This work applies classical, ensemble and neural network machine learning algorithms to a multispectral remote sensing dataset containing 405,000 28x28 pixel image patches in 4 electromagnetic frequency bands. For each algorithm, model metrics and prediction execution time were evaluated, resulting in two families of models; fast and precise. The prediction time for an 81,000-patch group of predictions wasmodels, and >5s for the precise models, and there was not a significant change in prediction time when a …

Diffusional Fractionation Of Helium Isotopes In Silicate Melts, Haiyang Luo, Bijaya Karki, Dipta B. Ghosh, Huiming Bao

Faculty Publications

Estimating Helium (He) concentration and isotope composition of the mantle requires quantifying He loss during magma degassing. The knowledge of diffusional He isotope fractionation in silicate melts may be essential to constrain the He loss. Isotopic mass dependence of He diffusion can be empirically expressed as D3He/D4He = (4/3)^β, where D is the diffusivity of a He isotope. However, no studies have reported any β values for He in silicate melts due to technical challenges in both experiments and computations. Here, molecular dynamics simulations based on deep neural network potentials trained by ab initio data …

Cognition-Enhanced Machine Learning For Better Predictions With Limited Data, Florian Sense, Ryan Wood, Michael G. Collins, Joshua Fiechter, Aihua W. Wood, Michael Krusmark, Tiffany Jastrzembski, Christopher W. Myers

Faculty Publications

The fields of machine learning (ML) and cognitive science have developed complementary approaches to computationally modeling human behavior. ML's primary concern is maximizing prediction accuracy; cognitive science's primary concern is explaining the underlying mechanisms. Cross-talk between these disciplines is limited, likely because the tasks and goals usually differ. The domain of e-learning and knowledge acquisition constitutes a fruitful intersection for the two fields’ methodologies to be integrated because accurately tracking learning and forgetting over time and predicting future performance based on learning histories are central to developing effective, personalized learning tools. Here, we show how a state-of-the-art ML model can …

Per-Pixel Cloud Cover Classification Of Multispectral Landsat-8 Data, Salome E. Carrasco [*], Torrey J. Wagner, Brent T. Langhals

Faculty Publications

Random forest and neural network algorithms are applied to identify cloud cover using 10 of the wavelength bands available in Landsat 8 imagery. The methods classify each pixel into 4 different classes: clear, cloud shadow, light cloud, or cloud. The first method is based on a fully connected neural network with ten input neurons, two hidden layers of 8 and 10 neurons respectively, and a single-neuron output for each class. This type of model is considered with and without L2 regularization applied to the kernel weighting. The final model type is a random forest classifier created from an ensemble of …

Synthetic Aperture Radar Image Recognition Of Armored Vehicles, Christopher Szul [*], Torrey J. Wagner, Brent T. Langhals

Faculty Publications

Synthetic Aperture Radar (SAR) imagery is not affected by weather and allows for day-and-night observations, however it can be difficult to interpret. This work applies classical and neural network machine learning techniques to perform image classification of SAR imagery. The Moving and Stationary Target Acquisition and Recognition dataset from the Air Force Research Laboratory was used, which contained 2,987 total observations of the BMP-2, BTR-70, and T-72 vehicles. Using a 75%/25% train/test split, the classical model achieved an average multi-class image recognition accuracy of 70%, while a convolutional neural network was able to achieve a 97% accuracy with lower model …

Defect Detection In Atomic Resolution Transmission Electron Microscopy Images Using Machine Learning, Philip Cho, Aihua W. Wood, Krishnamurthy Mahalingam, Kurt Eyink

Faculty Publications

Point defects play a fundamental role in the discovery of new materials due to their strong influence on material properties and behavior. At present, imaging techniques based on transmission electron microscopy (TEM) are widely employed for characterizing point defects in materials. However, current methods for defect detection predominantly involve visual inspection of TEM images, which is laborious and poses difficulties in materials where defect related contrast is weak or ambiguous. Recent efforts to develop machine learning methods for the detection of point defects in TEM images have focused on supervised methods that require labeled training data that is generated via …

Mlatticeabc: Generic Lattice Constant Prediction Of Crystal Materials Using Machine Learning, Yuxin Li, Wenhui Yang, Rongzhi Dong, Jianjun Hu

Faculty Publications

Lattice constants such as unit cell edge lengths and plane angles are important parameters of the periodic structures of crystal materials. Predicting crystal lattice constants has wide applications in crystal structure prediction and materials property prediction. Previous work has used machine learning models such as neural networks and support vector machines combined with composition features for lattice constant prediction and has achieved a maximum performance for cubic structures with an average coefficient of determination (R2) of 0.82. Other models tailored for special materials family of a fixed form such as ABX3 perovskites can achieve much higher performance due …

Acceleration Of Boltzmann Collision Integral Calculation Using Machine Learning, Ian Holloway, Aihua W. Wood, Alexander Alekseenko

Faculty Publications

The Boltzmann equation is essential to the accurate modeling of rarefied gases. Unfortunately, traditional numerical solvers for this equation are too computationally expensive for many practical applications. With modern interest in hypersonic flight and plasma flows, to which the Boltzmann equation is relevant, there would be immediate value in an efficient simulation method. The collision integral component of the equation is the main contributor of the large complexity. A plethora of new mathematical and numerical approaches have been proposed in an effort to reduce the computational cost of solving the Boltzmann collision integral, yet it still remains prohibitively expensive for …

A Physics-Based Machine Learning Study Of The Behavior Of Interstitial Helium In Single Crystal W–Mo Binary Alloys, Adib J. Samin

Faculty Publications

In this work, the behavior of dilute interstitial helium in W–Mo binary alloys was explored through the application of a first principles-informed neural network (NN) in order to study the early stages of helium-induced damage and inform the design of next generation materials for fusion reactors. The neural network (NN) was trained using a database of 120 density functional theory (DFT) calculations on the alloy. The DFT database of computed solution energies showed a linear dependence on the composition of the first nearest neighbor metallic shell. This NN was then employed in a kinetic Monte Carlo simulation, which took into …

Machine Learning Modeling Of Horizontal Photovoltaics Using Weather And Location Data, Christil Pasion, Torrey J. Wagner, Clay Koschnick, Steven J. Schuldt, Jada B. Williams, Kevin Hallinan

Faculty Publications

Solar energy is a key renewable energy source; however, its intermittent nature and potential for use in distributed systems make power prediction an important aspect of grid integration. This research analyzed a variety of machine learning techniques to predict power output for horizontal solar panels using 14 months of data collected from 12 northern-hemisphere locations. We performed our data collection and analysis in the absence of irradiation data—an approach not commonly found in prior literature. Using latitude, month, hour, ambient temperature, pressure, humidity, wind speed, and cloud ceiling as independent variables, a distributed random forest regression algorithm modeled the combined …

Cyber-Physical Security With Rf Fingerprint Classification Through Distance Measure Extensions Of Generalized Relevance Learning Vector Quantization, Trevor J. Bihl, Todd J. Paciencia, Kenneth W. Bauer Jr., Michael A. Temple

Faculty Publications

Radio frequency (RF) fingerprinting extracts fingerprint features from RF signals to protect against masquerade attacks by enabling reliable authentication of communication devices at the “serial number” level. Facilitating the reliable authentication of communication devices are machine learning (ML) algorithms which find meaningful statistical differences between measured data. The Generalized Relevance Learning Vector Quantization-Improved (GRLVQI) classifier is one ML algorithm which has shown efficacy for RF fingerprinting device discrimination. GRLVQI extends the Learning Vector Quantization (LVQ) family of “winner take all” classifiers that develop prototype vectors (PVs) which represent data. In LVQ algorithms, distances are computed between exemplars and PVs, and …

Critical Temperature Prediction Of Superconductors Based On Atomic Vectors And Deep Learning, Shaobo Li, Yabo Dan, Xiang Li, Tiantian Hu, Rongzhi Dong, Zhuo Cao, Jianjun Hu

Faculty Publications

In this paper, a hybrid neural network (HNN) that combines a convolutional neural network (CNN) and long short-term memory neural network (LSTM) is proposed to extract the high-level characteristics of materials for critical temperature (Tc) prediction of superconductors. Firstly, by obtaining 73,452 inorganic compounds from the Materials Project (MP) database and building an atomic environment matrix, we obtained a vector representation (atomic vector) of 87 atoms by singular value decomposition (SVD) of the atomic environment matrix. Then, the obtained atom vector was used to implement the coded representation of the superconductors in the order of the atoms in the chemical …

Heterogeneous Multi-Layered Network Model For Omics Data Integration And Analysis, Bohyun Lee, Shuo Zhang, Aleksandar Poleksic, Lei Xie

Faculty Publications

Advances in next-generation sequencing and high-throughput techniques have enabled the generation of vast amounts of diverse omics data. These big data provide an unprecedented opportunity in biology, but impose great challenges in data integration, data mining, and knowledge discovery due to the complexity, heterogeneity, dynamics, uncertainty, and high-dimensionality inherited in the omics data. Network has been widely used to represent relations between entities in biological system, such as protein-protein interaction, gene regulation, and brain connectivity (i.e. network construction) as well as to infer novel relations given a reconstructed network (aka link prediction). Particularly, heterogeneous multi-layered network (HMLN) has proven successful …

Analyze Informant-Based Questionnaire For The Early Diagnosis Of Senile Dementia Using Deep Learning, Fubao Zhu, Xiaonan Li, Daniel Mcgonigle, Haipeng Tang, Zhuo He, Chaoyang Zhang, Guang-Uei Hung, Pai-Yi Chu, Weihua Zhou

Faculty Publications

Objective: This paper proposes a multiclass deep learning method for the classification of dementia using an informant-based questionnaire.

Methods: A deep neural network classification model based on Keras framework is proposed in this paper. To evaluate the advantages of our proposed method, we compared the performance of our model with industry-standard machine learning approaches. We enrolled 6,701 individuals, which were randomly divided into training data sets (6030 participants) and test data sets (671 participants). We evaluated each diagnostic model in the test set using accuracy, precision, recall, and F1-Score.

Results: Compared with the seven conventional machine learning …

Machine Learning To Quantitate Neutrophil Netosis, Laila Elsherif, Noah Sciaky, Carrington A. Metts, Md. Modasshir, Ioannis Rekleitis, Christine A. Burris, Joshua A. Walker, Nadeem Ramadan, Tina M. Leisner, Stephen P. Holly, Martis W. Cowles, Kenneth I. Ataga, Joshua N. Cooper, Leslie V. Parise

Faculty Publications

We introduce machine learning (ML) to perform classifcation and quantitation of images of nuclei from human blood neutrophils. Here we assessed the use of convolutional neural networks (CNNs) using free, open source software to accurately quantitate neutrophil NETosis, a recently discovered process involved in multiple human diseases. CNNs achieved >94% in performance accuracy in diferentiating NETotic from non-NETotic cells and vastly facilitated dose-response analysis and screening of the NETotic response in neutrophils from patients. Using only features learned from nuclear morphology, CNNs can distinguish between NETosis and necrosis and between distinct NETosis signaling pathways, making them a precise tool for …

Fast And Effective Techniques For Lwir Radiative Transfer Modeling: A Dimension-Reduction Approach, Nicholas M. Westing [*], Brett J. Borghetti, Kevin C. Gross

Faculty Publications

The increasing spatial and spectral resolution of hyperspectral imagers yields detailed spectroscopy measurements from both space-based and airborne platforms. These detailed measurements allow for material classification, with many recent advancements from the fields of machine learning and deep learning. In many scenarios, the hyperspectral image must first be corrected or compensated for atmospheric effects. Radiative Transfer (RT) computations can provide look up tables (LUTs) to support these corrections. This research investigates a dimension-reduction approach using machine learning methods to create an effective sensor-specific long-wave infrared (LWIR) RT model.

Improving Optimization Of Convolutional Neural Networks Through Parameter Fine-Tuning, Nicholas C. Becherer, John M. Pecarina, Scott L. Nykl, Kenneth M. Hopkinson

Faculty Publications

In recent years, convolutional neural networks have achieved state-of-the-art performance in a number of computer vision problems such as image classification. Prior research has shown that a transfer learning technique known as parameter fine-tuning wherein a network is pre-trained on a different dataset can boost the performance of these networks. However, the topic of identifying the best source dataset and learning strategy for a given target domain is largely unexplored. Thus, this research presents and evaluates various transfer learning methods for fine-grained image classification as well as the effect on ensemble networks. The results clearly demonstrate the effectiveness of parameter …

Narrowing The Scope Of Failure Prediction Using Targeted Fault Load Injection, Paul L. Jordan, Gilbert L. Peterson, Alan C. Lin, Michael J. Mendenhall, Andrew J. Sellers

Faculty Publications

As society becomes more dependent upon computer systems to perform increasingly critical tasks, ensuring that those systems do not fail becomes increasingly important. Many organizations depend heavily on desktop computers for day-to-day operations. Unfortunately, the software that runs on these computers is written by humans and, as such, is still subject to human error and consequent failure. A natural solution is to use statistical machine learning to predict failure. However, since failure is still a relatively rare event, obtaining labelled training data to train these models is not a trivial task. This work presents new simulated fault-inducing loads that extend …

Unsupervised Machine Learning Account Of Magnetic Transitions In The Hubbard Model, Kelvin Ch'ng, Nick Vazquez, Ehsan Khatami

Faculty Publications

We employ several unsupervised machine learning techniques, including autoencoders, random trees embedding, and t-distributed stochastic neighboring ensemble (t-SNE), to reduce the dimensionality of, and therefore classify, raw (auxiliary) spin configurations generated, through Monte Carlo simulations of small clusters, for the Ising and Fermi-Hubbard models at finite temperatures. Results from a convolutional autoencoder for the three-dimensional Ising model can be shown to produce the magnetization and the susceptibility as a function of temperature with a high degree of accuracy. Quantum fluctuations distort this picture and prevent us from making such connections between the output of the autoencoder and …

Identification And Optimization Of Classifier Genes From Multi-Class Earthworm Microarray Dataset, Ying Li, Nan Wang, Chaoyang Zhang, Ping Gong

Faculty Publications

Monitoring, assessment and prediction of environmental risks that chemicals pose demand rapid and accurate diagnostic assays. A variety of toxicological effects have been associated with explosive compounds TNT and RDX. One important goal of microarray experiments is to discover novel biomarkers for toxicity evaluation. We have developed an earthworm microarray containing 15,208 unique oligo probes and have used it to profile gene expression in 248 earthworms exposed to TNT, RDX or neither. We assembled a new machine learning pipeline consisting of several well-established feature filtering/selection and classification techniques to analyze the 248-array dataset in order to construct classifier models that …

Malware Type Recognition And Cyber Situational Awareness, Thomas Dube, Richard A. Raines, Gilbert L. Peterson, Kenneth W. Bauer, Michael R. Grimaila, Steven K. Rogers

Faculty Publications

Current technologies for computer network and host defense do not provide suitable information to support strategic and tactical decision making processes. Although pattern-based malware detection is an active research area, the additional context of the type of malware can improve cyber situational awareness. This additional context is an indicator of threat capability thus allowing organizations to assess information losses and focus response actions appropriately. Malware Type Recognition (MaTR) is a research initiative extending detection technologies to provide the additional context of malware types using only static heuristics. Test results with MaTR demonstrate over a 99% accurate detection rate and 59% …

Developing Cyberspace Data Understanding Using Crisp-Dm For Host-Based Ids Feature Mining, Joseph R. Erskine, Gilbert L. Peterson, Barry E. Mullins, Michael R. Grimaila

Faculty Publications

Current intrusion detection systems (IDS) generate a large number of specific alerts, but typically do not provide actionable information. Compounding this problem is the fact that many alerts are false positive alerts. This paper applies the Cross Industry Standard Process for Data Mining (CRISP-DM) to develop an understanding of a host environment under attack. Data is generated by launching scans and exploits at a machine outfitted with a set of host-based forensic data collectors. Through knowledge discovery, features are selected to project human understanding of the attack process into the IDS model. By discovering relationships between the data collected and …

Scaling Ant Colony Optimization With Hierarchical Reinforcement Learning Partitioning, Erik J. Dries, Gilbert L. Peterson

Faculty Publications

This paper merges hierarchical reinforcement learning (HRL) with ant colony optimization (ACO) to produce a HRL ACO algorithm capable of generating solutions for large domains. This paper describes two specific implementations of the new algorithm: the first a modification to Dietterich’s MAXQ-Q HRL algorithm, the second a hierarchical ant colony system algorithm. These implementations generate faster results, with little to no significant change in the quality of solutions for the tested problem domains. The application of ACO to the MAXQ-Q algorithm replaces the reinforcement learning, Q-learning, with the modified ant colony optimization method, Ant-Q. This algorithm, MAXQ-AntQ, converges to solutions …