Engineering Commons^™

Tutorial - Shodhguru Labs: Optimization And Hyperparameter Tuning For Neural Networks, Kaushik Roy

Publications

Neural networks have emerged as a powerful and versatile class of machine learning models, revolutionizing various fields with their ability to learn complex patterns and make accurate predictions. The performance of neural networks depends significantly on the appropriate choice of hyperparameters, which are critical factors governing their architecture, regularization, and optimization techniques. As the demand for high-performance neural networks grows across diverse applications, the need for efficient optimization and hyperparameter tuning methods becomes paramount. This paper presents a comprehensive exploration of optimization strategies and hyperparameter tuning techniques for neural networks. Neural networks have emerged as a powerful and versatile class …

A Neural Network Based Proportional Hazard Model For Iot Signal Fusion And Failure Prediction, Yuxin Wen, Xingxin Guo, Junbo Son, Jianguo Wu

Engineering Faculty Articles and Research

Accurate prediction of remaining useful life (RUL) plays a critical role in optimizing condition-based maintenance decisions. In this paper, a novel joint prognostic modeling framework that simultaneously combines both time-to-event data and multi-sensor degradation signals is proposed. With the increasing use of IoT devices, unprecedented amounts of diverse signals associated with the underlying health condition of in-situ units have become easily accessible. To take full advantage of the modern IoT-enabled engineering systems, we propose a specialized framework for RUL prediction at the level of individual units. Specifically, a Bayesian linear regression model is developed for the multi-sensor degradation signals and …

Discrepancies Among Pre-Trained Deep Neural Networks: A New Threat To Model Zoo Reliability, Diego Montes, Pongpatapee Peerapatanapokin, Jeff Schultz, Chengjun Guo, Wenxin Jiang, James C. Davis

Department of Electrical and Computer Engineering Faculty Publications

Training deep neural networks (DNNs) takes significant time and resources. A practice for expedited deployment is to use pre-trained deep neural networks (PTNNs), often from model zoos.collections of PTNNs; yet, the reliability of model zoos remains unexamined. In the absence of an industry standard for the implementation and performance of PTNNs, engineers cannot confidently incorporate them into production systems. As a first step, discovering potential discrepancies between PTNNs across model zoos would reveal a threat to model zoo reliability. Prior works indicated existing variances in deep learning systems in terms of accuracy. However, broader measures of reliability for PTNNs from …

Artificial Intelligence And The Ethics Behind It, Isaac Johnston

Senior Honors Theses

Artificial intelligence (AI) has been a widely used buzzword for the past couple of years. If there is a technology that works without human interaction, it is labeled as AI. But what is AI, and should individuals be concerned? The following research aims to define what artificial intelligence is, specifically machine learning (ML) and neural networks. It is important to understand how AI is used today in cars, image recognition, ad marketing, and other areas. Although AI has many benefits, there are areas of ethical concerns such as autonomous cars, military applications, social media marketing, and others. This paper helps …

Machine Learning-Based Recognition On Crowdsourced Food Images, Aditya Kulkarni

Honors Scholar Theses

With nearly a third of the world’s population suffering from food-induced chronic diseases such as obesity, the role of food in community health is required now more than ever. While current research underscores food proximity and density, there is a dearth in regard to its nutrition and quality. However, recent research in geospatial data collection and analysis as well as intelligent deep learning will help us study this further.

Employing the efficiency and interconnection of computer vision and geospatial technology, we want to study whether healthy food in the community is attainable. Specifically, with the help of deep learning in …

Amodal Instance Segmentation And Multi-Object Tracking With Deep Pixel Embedding, Yanfeng Liu

Department of Electrical and Computer Engineering: Dissertations, Theses, and Student Research

This thesis extends upon the representational output of semantic instance segmentation by explicitly including both visible and occluded parts. A fully convolutional network is trained to produce consistent pixel-level embedding across two layers such that, when clustered, the results convey the full spatial extent and depth ordering of each instance. Results demonstrate that the network can accurately estimate complete masks in the presence of occlusion and outperform leading top-down bounding-box approaches.

The model is further extended to produce consistent pixel-level embeddings across two consecutive image frames from a video to simultaneously perform amodal instance segmentation and multi-object tracking. No post-processing …

Domain Adaptation In Unmanned Aerial Vehicles Landing Using Reinforcement Learning, Pedro Lucas Franca Albuquerque

Department of Computer Science and Engineering: Dissertations, Theses, and Student Research

Landing an unmanned aerial vehicle (UAV) on a moving platform is a challenging task that often requires exact models of the UAV dynamics, platform characteristics, and environmental conditions. In this thesis, we present and investigate three different machine learning approaches with varying levels of domain knowledge: dynamics randomization, universal policy with system identification, and reinforcement learning with no parameter variation. We first train the policies in simulation, then perform experiments both in simulation, making variations of the system dynamics with wind and friction coefficient, then perform experiments in a real robot system with wind variation. We initially expected that providing …

Data Analytics And Machine Learning To Enhance The Operational Visibility And Situation Awareness Of Smart Grid High Penetration Photovoltaic Systems, Aditya Sundararajan

FIU Electronic Theses and Dissertations

Electric utilities have limited operational visibility and situation awareness over grid-tied distributed photovoltaic systems (PV). This will pose a risk to grid stability when the PV penetration into a given feeder exceeds 60% of its peak or minimum daytime load. Third-party service providers offer only real-time monitoring but not accurate insights into system performance and prediction of productions. PV systems also increase the attack surface of distribution networks since they are not under the direct supervision and control of the utility security analysts.

Six key objectives were successfully achieved to enhance PV operational visibility and situation awareness: (1) conceptual cybersecurity …

Simulation Of Human Balance Control Using An Inverted Pendulum Model, Joshua E. Caneer

Undergraduate Research & Mentoring Program

The nervous system that human beings use to control balance is remarkably adaptable to a wide variety of environments and conditions. This neural system is likely a combination of many inputs and feedback control loops working together. The ability to emulate this system of balance could be of great value in understanding and developing solutions to proprioceptive disorders and other diseases that affect the human balance control system. Additionally, the process of emulating the human balance system may also have widespread applications to the locomotion capabilities of many types of robots, in both bipedal and non-bipedal configurations.

The goal of …

Depressiongnn: Depression Prediction Using Graph Neural Network On Smartphone And Wearable Sensors, Param Bidja

Honors Scholar Theses

Depression prediction is a complicated classification problem because depression diagnosis involves many different social, physical, and mental signals. Traditional classification algorithms can only reach an accuracy of no more than 70% given the complexities of depression. However, a novel approach using Graph Neural Networks (GNN) can be used to reach over 80% accuracy, if a graph can represent the depression data set to capture differentiating features. Building such a graph requires 1) the definition of node features, which must be highly correlated with depression, and 2) the definition for edge metrics, which must also be highly correlated with depression. In …

An Explainable Autoencoder For Collaborative Filtering Recommendation, Pegah Sagheb Haghighi, Olurotimi Seton, Olfa Nasraoui

Faculty Scholarship

Autoencoders are a common building block of Deep Learning architectures, where they are mainly used for representation learning. They have also been successfully used in Collaborative Filtering (CF) recommender systems to predict missing ratings. Unfortunately, like all black box machine learning models, they are unable to explain their outputs. Hence, while predictions from an Autoencoderbased recommender system might be accurate, it might not be clear to the user why a recommendation was generated. In this work, we design an explainable recommendation system using an Autoencoder model whose predictions can be explained using the neighborhood based explanation style. Our preliminary work …

Can Deep Learning Techniques Improve The Risk Adjusted Returns From Enhanced Indexing Investment Strategies, Anthony Grace

Dissertations

Deep learning techniques have been widely applied in the field of stock market prediction particularly with respect to the implementation of active trading strategies. However, the area of portfolio management and passive portfolio management in particular has been much less well served by research to date. This research project conducts an investigation into the science underlying the implementation of portfolio management strategies in practice focusing on enhanced indexing strategies. Enhanced indexing is a passive management approach which introduces an element of active management with the aim of achieving a level of active return through small adjustments to the portfolio weights. …

Neural Collaborative Filtering, Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

In recent years, deep neural networks have yielded immense success on speech recognition, computer vision and natural language processing. However, the exploration of deep neural networks on recommender systems has received relatively less scrutiny. In this work, we strive to develop techniques based on neural networks to tackle the key problem in recommendation --- collaborative filtering --- on the basis of implicit feedback.Although some recent work has employed deep learning for recommendation, they primarily used it to model auxiliary information, such as textual descriptions of items and acoustic features of musics. When it comes to model the key factor in …

Impact Of Reviewer Social Interaction On Online Consumer Review Fraud Detection, Kunal Goswami, Younghee Park, Chungsik Song

Faculty Publications

Background Online consumer reviews have become a baseline for new consumers to try out a business or a new product. The reviews provide a quick look into the application and experience of the business/product and market it to new customers. However, some businesses or reviewers use these reviews to spread fake information about the business/product. The fake information can be used to promote a relatively average product/business or can be used to malign their competition. This activity is known as reviewer fraud or opinion spam. The paper proposes a feature set, capturing the user social interaction behavior to identify fraud. …

Flexc: Protein Flexibility Prediction Using Context-Based Statistics, Predicted Structural Features, And Sequence Information, Ashraf Yaseen, Mais Nijim, Brandon Williams, Lei Qian, Min Li, Jianxin Wang, Yaohang Li

Computer Science Faculty Publications

The fluctuation of atoms around their average positions in protein structures provides important information regarding protein dynamics. This flexibility of protein structures is associated with various biological processes. Predicting flexibility of residues from protein sequences is significant for analyzing the dynamic properties of proteins which will be helpful in predicting their functions.

Coupled Auto-Associative Neural Networks For Heterogeneous Face Recognition, Benjamin S. Riggan, Christopher Reale, Nasser M. Nasrabadi

Department of Electrical and Computer Engineering: Faculty Publications

Several models have been previously suggested for learning correlated representations between source and target modalities. In this paper, we propose a novel coupled autoassociative neural network for learning a target-to-source image representation for heterogenous face recognition. This coupled network is unique, because a cross-modal transformation is learned by forcing the hidden units (latent features) of two neural networks to be as similar as possible, while simultaneously preserving information from the input. The effectiveness of this model is demonstrated using multiple existing heterogeneous face recognition databases. Moreover, the empirical results show that the learned image representation—common latent features—by the coupled auto-associative …

Orthogonal Bases Are The Best: A Theorem Justifying Bruno Apolloni's Heuristic Neural Network Idea, Jaime Nava, Vladik Kreinovich

Departmental Technical Reports (CS)

One of the main problems with neural networks is that they are often very slow in learning the desired dependence. To speed up neural networks, Bruno Apolloni proposed to othogonalize neurons during training, i.e., to select neurons whose output functions are orthogonal to each other. In this paper, we use symmetries to provide a theoretical explanation for this heuristic idea.

Towards Neural-Based Understanding Of The Cauchy Deviate Method For Processing Interval And Fuzzy Uncertainty, Vladik Kreinovich, Hung T. Nguyen

Departmental Technical Reports (CS)

One of the most efficient techniques for processing interval and fuzzy data is a Monte-Carlo type technique of Cauchy deviates that uses Cauchy distributions. This technique is mathematically valid, but somewhat counterintuitive. In this paper, following the ideas of Paul Werbos, we provide a natural neural network explanation for this technique.

Modified Art 2a Growing Network Capable Of Generating A Fixed Number Of Nodes, Ji He, Ah-Hwee Tan, Chew-Lim Tan

Research Collection School Of Computing and Information Systems

This paper introduces the Adaptive Resonance Theory under Constraint (ART-C 2A) learning paradigm based on ART 2A, which is capable of generating a user-defined number of recognition nodes through online estimation of an appropriate vigilance threshold. Empirical experiments compare the cluster validity and the learning efficiency of ART-C 2A with those of ART 2A, as well as three closely related clustering methods, namely online K-Means, batch K-Means, and SOM, in a quantitative manner. Besides retaining the online cluster creation capability of ART 2A, ART-C 2A gives the alternative clustering solution, which allows a direct control on the number of output …

Sensitivity Analysis Of Neural Control, Chin-Wang Tao, Hung T. Nguyen, J. T. Yao, Vladik Kreinovich

Departmental Technical Reports (CS)

We provide explicit formulas that describe how sensitive the resulting signal of a neural network is to the measurement errors with which we measure the inputs.

Computational Geometry And Artifical Neural Networks: A Hybrid Approach To Optimal Sensor Placement For Aerospace Nde, Roberto A. Osegueda, Carlos M. Ferregut, Mary J. George, Jose M. Gutierrez, Vladik Kreinovich

Departmental Technical Reports (CS)

The ideal design of an airplane should include built-in sensors that are pre-blended in the perfect aerodynamic shape. Each built-in sensor is expensive to blend in and requires continuous maintenance and data processing, so we would like to use as few sensors as possible. The ideal formulation of the corresponding optimization problem is, e.g., to minimize the average detection error for fault locations. However, there are two obstacles to this ideal formulation:

--First, this ideal formulation requires that we know the probabilities of different fault locations etc., and there are usually not enough statistics to determine these probabilities.

--Second, even …

Sensor Placement For Aerospace Non-Destructive Evaluation (Nde): Optimization Under Fuzzy Uncertainty, Roberto A. Osegueda, Carlos M. Ferregut, Mary J. George, Jose M. Gutierrez, Vladik Kreinovich

Departmental Technical Reports (CS)

No abstract provided.

Navigation Satellite Selection Using Neural Networks, Daniel J. Simon, Hossny El-Sherief

Electrical and Computer Engineering Faculty Publications

The application of neural networks to optimal satellite subset selection for navigation use is discussed. The methods presented in this paper are general enough to be applicable regardless of how many satellite signals are being processed by the receiver. The optimal satellite subset is chosen by minimizing a quantity known as Geometric Dilution of Precision (GDOP), which is given by the trace of the inverse of the measurement matrix. An artificial neural network learns the functional relationships between the entries of a measurement matrix and the eigenvalues of its inverse, and thus generates GDOP without inverting a matrix. Simulation results …