Engineering Commons^™

Optimized Learning Using Fuzzy-Inference-Assisted Algorithms For Deep Learning, Miroslava Barua

Open Access Theses & Dissertations

For years, researchers in Artificial Intelligence (AI) and Deep Learning (DL) observed that performance of a Deep Learning Network (DLN) could be improved by using larger and larger datasets coupled with complex network architectures. Although these strategies yield remarkable results, they have limits, dictated by data quantity and quality, rising costs by the increased computational power, or, more frequently, by long training times on networks that are very large. Training DLN requires laborious work involving multiple layers of densely connected neurons, updates to millions of network parameters, while potentially iterating thousands of times through millions of entries in a big …

Feed Forward Neural Networks With Asymmetric Training, Archit Srivastava

Department of Computer Science and Engineering: Dissertations, Theses, and Student Research

Our work presents a new perspective on training feed-forward neural networks(FFNN). We introduce and formally define the notion of symmetry and asymmetry in the context of training of FFNN. We provide a mathematical definition to generalize the idea of sparsification and demonstrate how sparsification can induce asymmetric training in FFNN.

In FFNN, training consists of two phases, forward pass and backward pass. We define symmetric training in FFNN as follows-- If a neural network uses the same parameters for both forward pass and backward pass, then the training is said to be symmetric.

The definition of asymmetric training in artificial …

Deep Learning Control For Digital Feedback Systems: Improved Performance With Robustness Against Parameter Change, Nuha A. S. Alwan, Zahir M. Hussain

Research outputs 2014 to 2021

Training data for a deep learning (DL) neural network (NN) controller are obtained from the input and output signals of a conventional digital controller that is designed to provide the suitable control signal to a specified plant within a feedback digital control system. It is found that if the DL controller is sufficiently deep (four hidden layers), it can outperform the conventional controller in terms of settling time of the system output transient response to a unit-step reference signal. That is, the DL controller introduces a damping effect. Moreover, it does not need to be retrained to operate with a …

Neural Networks Skin Tumor Diagnostic System, Zhao Zhang, William V. Stoecker, Randy Hays Moss

Zhao Zhang

In this study, a malignant melanoma diagnostic system is designed using a straightforward neural network with the back-propagation learning algorithm. Eleven features are automatically extracted from skin tumor images. The correct diagnostic rate of this system is better than the average rate of 16 dermatologists who based their diagnosis with only the slide images.

Application Of An Artificial Neural Network To Predict Graduation Success At The United States Military Academy, Gene Lesinski, Steven Corns, Cihan H. Dagli

Engineering Management and Systems Engineering Faculty Research & Creative Works

This paper presents a neural network approach to classify student graduation status based upon selected academic, demographic, and other indicators. A multi-layer feedforward network with backpropagation learning is used as the model framework. The model is trained, tested, and validated using 5100 student samples with data compiled from admissions records and institutional research databases. Nine input variables consist of categorical and numeric data elements including: high school rank, high school quality, standardized test scores, high school faculty assessments, extra-curricular activity score, parent's education status, and time since high school graduation. These inputs and the multi-layer neural network model are used …

Gender Classification: A Convolutional Neural Network Approach, Shan Sung Liew, Mohamed Khalil Hani, Syafeeza Ahmad Radzi, Rabia Bakhteri

Turkish Journal of Electrical Engineering and Computer Sciences

An approach using a convolutional neural network (CNN) is proposed for real-time gender classification based on facial images. The proposed CNN architecture exhibits a much reduced design complexity when compared with other CNN solutions applied in pattern recognition. The number of processing layers in the CNN is reduced to only four by fusing the convolutional and subsampling layers. Unlike in conventional CNNs, we replace the convolution operation with cross-correlation, hence reducing the computational load. The network is trained using a second-order backpropagation learning algorithm with annealed global learning rates. Performance evaluation of the proposed CNN solution is conducted on two …

Recognition System Of Indonesia Sign Language Based On Sensor And Artificial Neural Network, Endang Supriyati, Mohammad Iqbal

Makara Journal of Technology

Sign language as a kind of gestures is one of the most natural ways of communication for most people in deaf community. The aim of the sign language recognition is to provide a translation for sign gestures into meaningful text or speech so that communication between deaf and hearing society can easily be made. In this research, the Indonesian sign language recognition system based on flex sensors and an accelerometer is developed. This recognition system uses a sensory glove to capture data. The sensor data that are processed into feature vector are the 5-fingers bending and the palm acceleration when …

Controlling The Chaotic Discrete-Hénon System Using A Feedforward Neural Network With An Adaptive Learning Rate, Kürşad Gökce, Yilmaz Uyaroğlu

Turkish Journal of Electrical Engineering and Computer Sciences

This paper proposes a feedforward neural network-based control scheme to control the chaotic trajectories of a discrete-Hénon map in order to stay within an acceptable distance from the stable fixed point. An adaptive learning back propagation algorithm with online training is employed to improve the effectiveness of the proposed method. The simulation study carried in the discrete-Hénon system verifies the validity of the proposed control system.

Investigation Of The Divcon Neuron To Increase The Performance Of A Traditional Feed Forward Multi-Layer Perceptron And Its Hardware Implementation, Jovan Saenz

Open Access Theses & Dissertations

ABSTRACT

Artificial Neural Networks (ANNs) have been developed in an attempt to emulate the information processing capabilities of the biological brain. They offer an alternate computing approach to problems in which mathematical modeling is complicated, such as pattern recognition and pattern classification.

Since ANNs were proposed in the early 1940s, there has been a great amount of research effort dedicated to the development of new models that improve performance. Consequently, different architectures, a variety of activation functions, and distinct learning algorithms have been developed and implemented in different disciplines such as medicine, engineering, and science. In addition, ANNs have been …

Wide Area Monitoring In Power Systems Using Cellular Neural Networks, Bipul Luitel, Ganesh K. Venayagamoorthy

Electrical and Computer Engineering Faculty Research & Creative Works

The demand of power and the size and complexity of the power system is increasing. Wide area monitoring and control is an integral part in transitioning from the traditional power system to a Smart Grid. However, wide area monitoring becomes challenging as the size of the electric power grid, and consequently the number of components to be monitored, grows. Wide area monitor (WAM) designed using feed-forward and feedback neural network architectures do not scale up to handle the growing complexity of the Smart Grid. in this paper, cellular neural network (CNN) is presented as a way to provide scalability in …

Reconciling Motor Performance Indicators From Theoretical Calculations And Laboratory Tests, Himanshu Hirlekar, Badrul H. Chowdhury, Stephen Ruffing

Electrical and Computer Engineering Faculty Research & Creative Works

Quite often because of the complexity in the design of large industrial motors, the theoretical motor parameter calculations do not match actual results from laboratory tests. Thus, it becomes important to predict the amount of discrepancy between the two methods to develop confidence in the motor parameter calculations. This paper discusses the development of multiple artificial neural networks (ANNs) designed to predict the ratios of measured parameters to calculated parameters, given the geometry and construction of the motor. These ratios represent correction factors which can be applied to the values calculated from the theoretical program, which, in this case, is …

Neural Networks And The Natural Gradient, Michael R. Bastian

All Graduate Theses and Dissertations, Spring 1920 to Summer 2023

Neural network training algorithms have always suffered from the problem of local minima. The advent of natural gradient algorithms promised to overcome this shortcoming by finding better local minima. However, they require additional training parameters and computational overhead. By using a new formulation for the natural gradient, an algorithm is described that uses less memory and processing time than previous algorithms with comparable performance.

A Quantum Calculus Formulation Of Dynamic Programming And Ordered Derivatives, John E. Seiffertt Iv, Donald C. Wunsch

Electrical and Computer Engineering Faculty Research & Creative Works

Much recent research activity has focused on the theory and application of quantum calculus. This branch of mathematics continues to find new and useful applications and there is much promise left for investigation into this field. We present a formulation of dynamic programming grounded in the quantum calculus. Our results include the standard dynamic programming induction algorithm which can be interpreted as the Hamilton-Jacobi-Bellman equation in the quantum calculus. Furthermore, we show that approximate dynamic programming in quantum calculus is tenable by laying the groundwork for the backpropagation algorithm common in neural network training. In particular, we prove that the …

Combined Training Of Recurrent Neural Networks With Particle Swarm Optimization And Backpropagation Algorithms For Impedance Identification, Peng Xiao, Ganesh K. Venayagamoorthy, Keith Corzine

Electrical and Computer Engineering Faculty Research & Creative Works

A recurrent neural network (RNN) trained with a combination of particle swarm optimization (PSO) and backpropagation (BP) algorithms is proposed in this paper. The network is used as a dynamic system modeling tool to identify the frequency-dependent impedances of power electronic systems such as rectifiers, inverters, and DC-DC converters. As a category of supervised learning methods, the various backpropagation training algorithms developed for recurrent neural networks use gradient descent information to guide their search for optimal weights solutions that minimize the output errors. While they prove to be very robust and effective in training many types of network structures, they …

A Novel Method For Predicting Harmonic Current Injection From Non-Linear Loads Using Neural Networks, Joy Mazumdar, Frank C. Lambert, Ganesh K. Venayagamoorthy, Ronald G. Harley

Electrical and Computer Engineering Faculty Research & Creative Works

Generation of harmonics and the existence of waveform pollution in power system networks is one of the major problems facing the utilities. This paper proposes a neural network solution methodology for the problem of measuring the actual amount of harmonic current injected into a power network by a nonlinear load. The determination of harmonic currents is complicated by the fact that the supply voltage waveform is distorted by other loads and is rarely a pure sinusoid. A recurrent neural network trained with the backpropagation through time (BPTT) training algorithm is used to find a way of distinguishing between the load …

A Comparison Of Pso And Backpropagation For Training Rbf Neural Networks For Identification Of A Power System With Statcom, Salman Mohagheghi, Ganesh K. Venayagamoorthy, Yamille Del Valle, Ronald G. Harley

Electrical and Computer Engineering Faculty Research & Creative Works

Backpropagation algorithm is the most commonly used algorithm for training artificial neural networks. While being a straightforward procedure, it suffers from extensive computations, relatively slow convergence speed and possible divergence for certain conditions. The efficiency of this method as the training algorithm of a radial basis function neural network (RBFN) is compared with that of particle swarm optimization, for neural network based identification of a small power system with a static compensator. The comparison of the two methods is based on the convergence speed and robustness of each method.

Optimal Dynamic Neurocontrol Of A Gate-Controlled Series Capacitor In A Multi-Machine Power System, Swakshar Ray, Ganesh K. Venayagamoorthy, Edson H. Watanabe, F. D. De Jesus

Electrical and Computer Engineering Faculty Research & Creative Works

This paper presents the design of an optimal dynamic neurocontroller for a new type of FACTS device - the gate controlled series capacitor (GCSC) incorporated in a multi-machine power system. The optimal neurocontroller is developed based on the heuristic dynamic programming (HDP) approach. In addition, a dynamic identifier/model and controller structure using the recurrent neural network trained with backpropagation through time (BPTT) is employed. Simulation results are presented to show the effectiveness of the dynamic neurocontroller and its performance is compared with that of the conventional PI controller under small and large disturbances.

Inference Of Genetic Regulatory Networks With Recurrent Neural Network Models, Rui Xu, Xiao Hu, Donald C. Wunsch

Electrical and Computer Engineering Faculty Research & Creative Works

Large-scale gene expression data coming from microarray experiments provide us a new means to reveal fundamental cellular processes, investigate functions of genes, and understand relations and interactions among them. To infer genetic regulatory networks from these data with effective computational tools has become increasingly important Several mathematical models, including Boolean networks, Bayesian networks, dynamic Bayesian networks, and linear additive regulation models, have been used to explore the behaviors of regulatory networks. In this paper, we investigate the inference of genetic regulatory networks from time series gene expression in the framework of recurrent neural network model.

Time Series Prediction With A Weighted Bidirectional Multi-Stream Extended Kalman Filter, Donald C. Wunsch, Xiao Hu

Electrical and Computer Engineering Faculty Research & Creative Works

This paper describes the use of a multi-stream extended Kalman filter (EKF) to tackle the IJCNN 2004 challenge problem - time series prediction on CATS benchmark. A weighted bidirectional approach was adopted in the experiments to incorporate the forward and backward predictions of the time series. EKF is a practical, general approach to neural networks training. It consists of the following: 1) gradient calculation by backpropagation through time (BPTT); 2) weight updates based on the extended Kalman filter; and 3) data presentation using multi-stream mechanics.

Neural Network Based Classification Of Road Pavement Structures, V. Venayagamoorthy, D. Allopi, Ganesh K. Venayagamoorthy

Electrical and Computer Engineering Faculty Research & Creative Works

Roads have formed the basic infrastructure of commerce since flints and other tools and artifacts were first exchanged along the trade routes of prehistory. Roadways are very large, in volume, in extent, and in value. They also wear out, and their useful life is directly proportional to their initial strength and inversely proportional to the number of heavy goods vehicles using them. Therefore, the increasing complexity of road transportation needs advanced techniques for effective design of pavements. This paper proposes an intelligent technique using neural networks to classify different types of road pavement structures, which is essential in estimating bearing …

Neuro-Fuzzy Approach For Development Of New Neuron Model, Manmohan, D. K. Chaturvedi, P. S. Satsangi, P. K. Kalra

D. K. Chaturvedi Dr.

The training time of ANN depends on size of ANN (i.e. number of hidden layers and number of neurons in each layer), size of training data, their normalization range and type of mapping of training patterns (like X–Y, X–DY, DX–Y and DX–DY), error functions and learning algorithms. The efforts have been done in past to reduce training time of ANN by selection of an optimal network and modification in learning algorithms. In this paper, an attempt has been made to develop a new neuron model using neuro-fuzzy approach to overcome the problems of ANN incorporating the features of fuzzy systems …

Intelligent Strain Sensing On A Smart Composite Wing Using Extrinsic Fabry-Perot Interferometric Sensors And Neural Networks, Kakkattukuzhy M. Isaac, Donald C. Wunsch, Steve Eugene Watkins, Rohit Dua, V. M. Eller

Electrical and Computer Engineering Faculty Research & Creative Works

Strain prediction at various locations on a smart composite wing can provide useful information on its aerodynamic condition. The smart wing consisted of a glass/epoxy composite beam with three extrinsic Fabry-Perot interferometric (EFPI) sensors mounted at three different locations near the wing root. Strain acting on the three sensors at different air speeds and angles-of-attack were experimentally obtained in a closed circuit wind tunnel under normal conditions of operation. A function mapping the angle of attack and air speed to the strains on the three sensors was simulated using feedforward neural networks trained using a backpropagation training algorithm. This mapping …

Neural Networks Skin Tumor Diagnostic System, Zhao Zhang, William V. Stoecker, Randy Hays Moss

Electrical and Computer Engineering Faculty Research & Creative Works

In this study, a malignant melanoma diagnostic system is designed using a straightforward neural network with the back-propagation learning algorithm. Eleven features are automatically extracted from skin tumor images. The correct diagnostic rate of this system is better than the average rate of 16 dermatologists who based their diagnosis with only the slide images.

Comparison Of Particle Swarm Optimization And Backpropagation As Training Algorithms For Neural Networks, Ganesh K. Venayagamoorthy, Venu Gopal Gudise

Electrical and Computer Engineering Faculty Research & Creative Works

Particle swarm optimization (PSO) motivated by the social behavior of organisms, is a step up to existing evolutionary algorithms for optimization of continuous nonlinear functions. Backpropagation (BP) is generally used for neural network training. Choosing a proper algorithm for training a neural network is very important. In this paper, a comparative study is made on the computational requirements of the PSO and BP as training algorithms for neural networks. Results are presented for a feedforward neural network learning a nonlinear function and these results show that the feedforward neural network weights converge faster with the PSO than with the BP …

Comparison Of Heuristic Dynamic Programming And Dual Heuristic Programming Adaptive Critics For Neurocontrol Of A Turbogenerator, Ganesh K. Venayagamoorthy, Donald C. Wunsch, Ronald G. Harley

Electrical and Computer Engineering Faculty Research & Creative Works

This paper presents the design of an optimal neurocontroller that replaces the conventional automatic voltage regulator (AVR) and the turbine governor for a turbogenerator connected to the power grid. The neurocontroller design uses a novel technique based on the adaptive critic designs (ACDs), specifically on heuristic dynamic programming (HDP) and dual heuristic programming (DHP). Results show that both neurocontrollers are robust, but that DHP outperforms HDP or conventional controllers, especially when the system conditions and configuration change. This paper also shows how to design optimal neurocontrollers for nonlinear systems, such as turbogenerators, without having to do continually online training of …

Comparison Of Mlp And Rbf Neural Networks Using Deviation Signals For Indirect Adaptive Control Of A Synchronous Generator, Jung-Wook Park, Ganesh K. Venayagamoorthy, Ronald G. Harley

Electrical and Computer Engineering Faculty Research & Creative Works

This paper compares the performances of a multilayer perceptron neurocontroller and a radial basis function neurocontroller for backpropagation through time based indirect adaptive control of the synchronous generator. Also, the neurocontrollers are compared with the conventional controller for small as well as large disturbances to the power system

Detection And Classification Of Impact-Induced Damage In Composite Plates Using Neural Networks, Rohit Dua, Steve Eugene Watkins, Donald C. Wunsch, K. Chandrashekhara, Farhad Akhavan

Electrical and Computer Engineering Faculty Research & Creative Works

Artificial neutral networks (ANN) can be used as an online health monitoring systems (involving damage assessment, fatigue monitoring and delamination detection) for composite structures owing to their inherent fast computing speeds, parallel processing and ability to learn and adapt to the experimental data. The amount of impact-induced strain on a composite structure can be found using strain sensors attached to composite structures. Prior work has shown that strain-based ANN can characterize impact energy on composite plates and that strain signatures can be associated with damage types and severity. This paper reports the extension of this approach for damage classification using …

A Nonlinear Voltage Controller With Derivative Adaptive Critics For Multimachine Power Systems, Ganesh K. Venayagamoorthy, Donald C. Wunsch, Ronald G. Harley

Electrical and Computer Engineering Faculty Research & Creative Works

Based on derivative adaptive critics, a novel nonlinear optimal voltage/excitation control for a multimachine power system is presented. The feedback variables are completely based on local measurements. Simulations on a three-machine system demonstrate that the nonlinear controller is much more effective than the conventional PID controller equipped with a power system stabilizer for improving dynamic performance and stability under small and large disturbances.

Comparison Of A Heuristic Dynamic Programming And A Dual Heuristic Programming Based Adaptive Critics Neurocontroller For A Turbogenerator, Ganesh K. Venayagamoorthy, Donald C. Wunsch, Ronald G. Harley

Electrical and Computer Engineering Faculty Research & Creative Works

This paper presents the design of a neurocontroller for a turbogenerator that augments/replaces the conventional automatic voltage regulator and the turbine governor. The neurocontroller uses a novel technique based on the adaptive critic designs with emphasis on heuristic dynamic programming (HDP) and dual heuristic programming (DHP). Results are presented to show that the DHP based neurocontroller is robust and performs better than the HDP based neurocontroller, as well as the conventional controller, especially when the system conditions and configuration changes.

Neurocontroller Alternatives For "Fuzzy" Ball-And-Beam Systems With Nonuniform Nonlinear Friction, Danil V. Prokhorov, Donald C. Wunsch, Paul H. Eaton

Electrical and Computer Engineering Faculty Research & Creative Works

The ball-and-beam problem is a benchmark for testing control algorithms. Zadeh proposed (1994) a twist to the problem, which, he suggested, would require a fuzzy logic controller. This experiment uses a beam, partially covered with a sticky substance, increasing the difficulty of predicting the ball's motion. We complicated this problem even more by not using any information concerning the ball's velocity. Although it is common to use the first differences of the ball's consecutive positions as a measure of velocity and explicit input to the controller, we preferred to exploit recurrent neural networks, inputting only consecutive positions instead. We have …