Computer Sciences Commons^™

Automatic Source Attribution Of Text: A Neural Networks Approach, Foaad Khosmood, Franz J. Kurfess

Computer Science and Software Engineering

Recent advances in automatic authorship attribution have been promising. Relatively new techniques such as N-gram analysis have shown important improvements in accuracy [2]. Much of the work in this area does remain in the realm of statistics best suited for human assistance rather than autonomous attribution [6]. While there have been attempts at using neural networks in the area in the past, they have been extremely limited and problem-specific [7]. This paper addresses the latter points by demonstrating a practical and truly autonomous attribution process using neural networks. Furthermore, we use a word-frequency classification technique to demonstrate the feasibility of …

Effectively Using Recurrently-Connected Spiking Neural Networks, Eric Goodman, Dan A. Ventura

Faculty Publications

Recurrently-connected spiking neural networks are difficult to use and understand because of the complex nonlinear dynamics of the system. Through empirical studies of spiking networks, we deduce several principles which are critical to success. Network parameters such as synaptic time delays and time constants and the connection probabilities can be adjusted to have a significant impact on accuracy. We show how to adjust these parameters to fit the type of problem.

On The Optimization Of Visualizations Of Complex Phenomena, Donald H. House, Althea D. Bair, Colin Ware

Center for Coastal and Ocean Mapping

The problem of perceptually optimizing complex visualizations is a difficult one, involving perceptual as well as aesthetic issues. In our experience, controlled experiments are quite limited in their ability to uncover interrelationships among visualization parameters, and thus may not be the most useful way to develop rules-of-thumb or theory to guide the production of high-quality visualizations. In this paper, we propose a new experimental approach to optimizing visualization quality that integrates some of the strong points of controlled experiments with methods more suited to investigating complex highly-coupled phenomena. We use human-in-the-loop experiments to search through visualization parameter space, generating large …

Modified Art 2a Growing Network Capable Of Generating A Fixed Number Of Nodes, Ji He, Ah-Hwee Tan, Chew-Lim Tan

Research Collection School Of Computing and Information Systems

This paper introduces the Adaptive Resonance Theory under Constraint (ART-C 2A) learning paradigm based on ART 2A, which is capable of generating a user-defined number of recognition nodes through online estimation of an appropriate vigilance threshold. Empirical experiments compare the cluster validity and the learning efficiency of ART-C 2A with those of ART 2A, as well as three closely related clustering methods, namely online K-Means, batch K-Means, and SOM, in a quantitative manner. Besides retaining the online cluster creation capability of ART 2A, ART-C 2A gives the alternative clustering solution, which allows a direct control on the number of output …

A Generalised Feedforward Neural Network Architecture And Its Applications To Classification And Regression, Ganesh Arulampalam

Theses: Doctorates and Masters

Shunting inhibition is a powerful computational mechanism that plays an important role in sensory neural information processing systems. It has been extensively used to model some important visual and cognitive functions. It equips neurons with a gain control mechanism that allows them to operate as adaptive non-linear filters. Shunting Inhibitory Artificial Neural Networks (SIANNs) are biologically inspired networks where the basic synaptic computations are based on shunting inhibition. SIANNs were designed to solve difficult machine learning problems by exploiting the inherent non-linearity mediated by shunting inhibition. The aim was to develop powerful, trainable networks, with non-linear decision surfaces, for classification …

Contrast Enhancement Of Ultrasound Images Using Shunting Inhibitory Cellular Neural Networks, Murali M. Gogineni

Theses: Doctorates and Masters

Evolving from neuro-biological insights, neural network technology gives a computer system an amazing capacity to actually generate decisions dynamically. However, as the amount of data to be processed increases, there is a demand for developing new types of networks such as Cellular Neural Networks (CNN), to ease the computational burden without compromising the outcomes. The objective of this thesis is to research the capability of Shunting Inhibitory Cellular Neural Networks (SICNN) to solve the clarity problems in ultrasound imaging. In this thesis, we begin by reviewing a number of traditional enhancement techniques and measures. Since the entire work of this …

High Performance Data Mining Techniques For Intrusion Detection, Muazzam Ahmed Siddiqui

Electronic Theses and Dissertations

The rapid growth of computers transformed the way in which information and data was stored. With this new paradigm of data access, comes the threat of this information being exposed to unauthorized and unintended users. Many systems have been developed which scrutinize the data for a deviation from the normal behavior of a user or system, or search for a known signature within the data. These systems are termed as Intrusion Detection Systems (IDS). These systems employ different techniques varying from statistical methods to machine learning algorithms. Intrusion detection systems use audit data generated by operating systems, application softwares or …

Enabling Computer Decisions Based On Eeg Input, Benjamin J. Culpepper, Robert M. Keller

All HMC Faculty Publications and Research

Multilayer neural networks were successfully trained to classify segments of 12-channel electroencephalogram (EEG) data into one of five classes corresponding to five cognitive tasks performed by a subject. Independent component analysis (ICA) was used to segregate obvious artifact EEG components from other sources, and a frequency-band representation was used to represent the sources computed by ICA. Examples of results include an 85% accuracy rate on differentiation between two tasks, using a segment of EEG only 0.05 s long and a 95% accuracy rate using a 0.5-s-long segment.

Simplifying Ocr Neural Networks With Oracle Learning, Tony R. Martinez, Joshua Menke

Faculty Publications

Often the best model to solve a real world problem is relatively complex. The following presents oracle learning, a method using a larger model as an oracle to train a smaller model on unlabeled data in order to obtain (1) a simpler acceptable model and (2) improved results over standard training methods on a similarly sized smaller model. In particular, this paper looks at oracle learning as applied to multi-layer perceptrons trained using standard backpropagation. For optical character recognition, oracle learning results in an 11.40% average decrease in error over direct training while maintaining 98.95% of the initial oracle accuracy.

Concurrently Learning Neural Nets: Encouraging Optimal Behavior In Cooperative Reinforcement Learning Systems, Nancy Fulda, Dan A. Ventura

Faculty Publications

Reinforcement learning agents interacting in a common environment often fail to converge to optimal system behaviors even when the individual goals of the agents are fully compatible. Claus and Boutilier have demonstrated that the use of joint action learning helps to overcome these difficulties for Q-learning systems. This paper studies an application of joint action learning to systems of neural networks. Neural networks are a desirable candidate for such augmentations for two reasons: (1) they may be able to generalize more effectively than Q-learners, and (2) the network topology used may improve the scalability of joint action learning to systems …

Machine Learning Approaches For Determining Effective Seeds For K -Means Algorithm, Kaveephong Lertwachara

Doctoral Dissertations

In this study, I investigate and conduct an experiment on two-stage clustering procedures, hybrid models in simulated environments where conditions such as collinearity problems and cluster structures are controlled, and in real-life problems where conditions are not controlled. The first hybrid model (NK) is an integration between a neural network (NN) and the k-means algorithm (KM) where NN screens seeds and passes them to KM. The second hybrid (GK) uses a genetic algorithm (GA) instead of the neural network. Both NN and GA used in this study are in their simplest-possible forms.

In the simulated data sets, I investigate two …

Automatic Speaker Identification Using Reusable And Retrainable Binary-Pair Partitioned Neural Networks, Ashutosh Mishra

Electrical & Computer Engineering Theses & Dissertations

This thesis presents an extension of the work previously done on speaker identification using Binary Pair Partitioned (BPP) neural networks. In the previous work, a separate network was used for each pair of speakers in the speaker population. Although the basic BPP approach did perform well and had a simple underlying algorithm, it had the obvious disadvantage of requiring an extremely large number of networks for speaker identification with large speaker populations. It also requires training of networks proportional to the square of the number of speakers under consideration, leading to a very large number of networks to be trained …

A Memory-Based Approach To Cantonese Tone Recognition, Deryle W. Lonsdale, Michael Emonts

Faculty Publications

This paper introduces memory-based learning as a viable approach for Cantonese tone recognition. The memorybased learning algorithm employed here outperforms other documented current approaches for this problem, which is based on neural networks. Various numbers of tones and features are modeled to find the best method for feature selection and extraction. To further optimize this approach, experiments are performed to isolate the best feature weighting method, the best class voting weights method, and the best number of k-values to implement. Results and possible future work are discussed.

Data Mining With Newton's Method., James Dale Cloyd

Electronic Theses and Dissertations

Capable and well-organized data mining algorithms are essential and fundamental to helpful, useful, and successful knowledge discovery in databases. We discuss several data mining algorithms including genetic algorithms (GAs). In addition, we propose a modified multivariate Newton's method (NM) approach to data mining of technical data. Several strategies are employed to stabilize Newton's method to pathological function behavior. NM is compared to GAs and to the simplex evolutionary operation algorithm (EVOP). We find that GAs, NM, and EVOP all perform efficiently for well-behaved global optimization functions with NM providing an exponential improvement in convergence rate. For local optimization problems, we …

Recurrent Neural Networks And Algorithms For Reconstruction Of Images From Noisy And/Or Partial Data, Ming-Jung Seow

Electrical & Computer Engineering Theses & Dissertations

In this thesis, modular architectures and neighborhood-distance based learning algorithms for fast and effective convergence with increased storage capacity of Hopfield neural networks are presented. The main objective of this research work is to better understand the function of recurrent neural networks and the influence of modularity within a network, and to design, implement, and test the performance of modular Hopfield neural networks for pattern association. Mathematical analysis and results are provided to show that the speed, storage capacity, and generalization capability of the recurrent networks are improved significantly by incorporating the modular architectures and learning algorithms. A new ratio …

Improving Speech Recognition Learning Through Lazy Training, Tony R. Martinez, Michael E. Rimer, D. Randall Wilson

Faculty Publications

Multi-layer backpropagation, like most learning algorithms that can create complex decision surfaces, is prone to overfitting. We present a novel approach, called lazy training, for reducing the overfit in multiple-layer networks. Lazy training consistently reduces generalization error of optimized neural networks by more than half on a large OCR dataset and on several real world problems from the UCI machine learning database repository. Here, lazy training is shown to be effective in a multi-layered adaptive learning system, reducing the error of an optimized backpropagation network in a speech recognition system by 50.0% on the TIDIGITS corpus.

The Effect Of Model Formulation On The Comparative Performance Of Artificial Neural Networks And Regression, Michael F. Cochrane

Engineering Management & Systems Engineering Theses & Dissertations

Multiple linear regression techniques have been traditionally used to construct predictive statistical models, relating one or more independent variables (inputs) to a dependent variable (output). Artificial neural networks can also be constructed and trained to learn these complex relationships, and have been shown to perform at least as well as linear regression on the same data sets. Research on the use of neural network models as alternatives to multivariate linear regression has focused predominantly on the effects of sample size, noise, and input vector size on the comparative performance of these two modeling techniques. However, research has also shown that …

Predictive Self-Organizing Networks For Text Categorization, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

This paper introduces a class of predictive self-organizing neural networks known as Adaptive Resonance Associative Map (ARAM) for classification of free-text documents. Whereas most sta- tistical approaches to text categorization derive classification knowledge based on training examples alone, ARAM performs supervised learn- ing and integrates user-defined classification knowledge in the form of IF-THEN rules. Through our experiments on the Reuters-21578 news database, we showed that ARAM performed reasonably well in mining categorization knowledge from sparse and high dimensional document feature space. In addition, ARAM predictive accuracy and learning efficiency can be improved by incorporating a set of rules derived from …

Fingerprint Recognition Using Neural Networks, Eng Hoe Kennie Yeoh

Student Works (2000-2009)

Traditional methods of fingerprint verification uses either complicated feature detection algorithms that are not specific to each fingerprint, or compare two fingerprint images directly using image processing toots. The former involves very complicated calculations and tedious algorithms, and the latter tend to work poorly. In this paper it is described a new method which takes the middle ground. This paper studies the implementation of the Fast Fourier Transform and Artificial Neural Networks into the recognition of fingerprints. With tests conducted on the implementation of the Fourier Transform as a method of fingerprint feature extraction, the use of the Fourier Transform …

Decision Support Methods In Diabetic Patient Management By Insulin Administration Neural Network Vs. Induction Methods For Knowledge Classification, B. V. Ambrosiadou, S. Vadera, Venky Shankaraman, D. Goulis, G. Gogou

Research Collection School Of Computing and Information Systems

Diabetes mellitus is now recognised as a major worldwide public health problem. At present, about 100 million people are registered as diabetic patients. Many clinical, social and economic problems occur as a consequence of insulin-dependent diabetes. Treatment attempts to prevent or delay complications by applying ‘optimal’ glycaemic control. Therefore, there is a continuous need for effective monitoring of the patient. Given the popularity of decision tree learning algorithms as well as neural networks for knowledge classification which is further used for decision support, this paper examines their relative merits by applying one algorithm from each family on a medical problem; …

Neural Networks And Structured Knowledge: Rule Extraction And Applications, Franz J. Kurfess

Computer Science and Software Engineering

As the second part of a special issue on "Neural Networks and Structured Knowledge," the contributions collected here concentrate on the extraction of knowledge, particularly in the form of rules, from neural networks, and on applications relying on the representation and processing of structured knowledge by neural networks. The transformation of the low-level internal representation in a neural network into higher-level knowledge or information that can be interpreted more easily by humans and integrated with symbol-oriented mechanisms is the subject of the first group of papers. The second group of papers uses specific applications as starting point, and describes approaches …

Neural Networks And Structured Knowledge: Knowledge Representation And Reasoning, Franz J. Kurfess

Computer Science and Software Engineering

This collection of articles is the first of two parts of a special issue on "Neural Networks and Structured Knowledge." The contributions to the first part shed some light on the issues of knowledge representation and reasoning with neural networks. Their scope ranges from formal models for mapping discrete structures like graphs or logical formulae onto different types of neural networks, to the construction of practical systems for various types of reasoning. In the second part to follow, the emphasis will be on the extraction of knowledge from neural networks, and on applications of neural networks and structured knowledge to …

Newton Parameter Update Algorithm For Recurrent Neural Networks Applied To Adaptive System Identification And Control, Donald Allen Gates

Electrical & Computer Engineering Theses & Dissertations

This paper shows that the combination of a second-order neural network parameter update algorithm and internal network feedback can be effectively used for adaptive, nonlinear, dynamical system identification and control. Adaptive neural identification and control algorithms are typically utilized for real-time applications where the rate of adaptation is often critical. A fast, adaptive network parameter update algorithm is presented.

Simulation results show that this algorithm is capable of quickly identifying and adapting to changes in system parameters, making it feasible to use for real-time control and fault accommodation applications.

An Adaptive Hierarchical Fuzzy Logic System For Modelling And Prediction Of Financial Systems, Mark Kingham

Theses: Doctorates and Masters

In this thesis, an intelligent fuzzy logic system using genetic algorithms for the prediction and modelling of interest rates is developed. The proposed system uses a Hierarchical Fuzzy Logic system in which a genetic algorithm is used as a training method for learning the fuzzy rules knowledge bases. A fuzzy logic system is developed to model and predict three month quarterly interest rate fluctuations. The system is further trained to model and predict interest rates for six month and one year periods. The proposed system is developed with first two, three, then four and finally five hierarchical knowledge bases to …

Study Of Human Factors Variables In Battle Outcome Prediction Models, David Andrew Glovier

Engineering Management & Systems Engineering Theses & Dissertations

Over time there have been many improvements in models that are used to predict the outcome of battles. Currently there is much supposition and speculation surrounding the use of human performance related factors as additional inputs to battle simulation models to improve their accuracy. However there is no conclusive scientific evidence which shows that these factors do make a significant difference. This study investigates the use of factors that may impact on the human performance directly or indirectly in battle prediction models. These factors consist of traditional human factors and external factors that may influence the human performance. The research …

Robust Optimization Using Training Set Evolution, Tony R. Martinez, Dan A. Ventura

Faculty Publications

Training Set Evolution is an eclectic optimization technique that combines evolutionary computation (EC) with neural networks (NN). The synthesis of EC with NN provides both initial unsupervised random exploration of the solution space as well as supervised generalization on those initial solutions. An assimilation of a large amount of data obtained over many simulations provides encouraging empirical evidence for the robustness of Evolutionary Training Sets as an optimization technique for feedback and control problems.

Generalization Metrics For Neural Modeling Applications In System Identification, Denise M. Reeves

Electrical & Computer Engineering Theses & Dissertations

In this thesis a procedure to design multilayer feedforward networks for system identification with good prediction properties is presented. Central to the design procedure is a means to characterize the prediction capabilities of various trained neural networks. Such knowledge will allow for the identification of the best network design. For system identification purposes, a "good" model is one that is good at predicting, In particular, a good model is one that produces small prediction errors when applied to a set of cross-validation data. We formulate and implement a criterion function designed to measure the size of a trained neural network's …

Multiprocessor Document Allocation: A Neural Network Approach, Abdulaziz Sultan Al-Sehibani, Kishan Mehrotra, Chilukuri K. Mohan, Sanjay Ranka

Electrical Engineering and Computer Science - Technical Reports

We consider the problem of distributing the documents to a given set of processors so that the load on each processor is as equal as possible and the amount of communication is as small as possible. This is an NP-Complete problem. We apply continuous as well as discrete Hopfield neural networks to obtain suboptimal solutions for the problem. These networks perform better than a genetic algorithm for this task proposed by Frieder et al. [4]; in particular, the continuous Hopfield network performs extremely well.

A Recursive Least Squares Training Algorithm For Multilayer Recurrent Neural Networks, Q. Xu, K. Krishnamurthy, Bruce M. Mcmillin, Wen Feng Lu

Mechanical and Aerospace Engineering Faculty Research & Creative Works

Recurrent neural networks have the potential to perform significantly better than the commonly used feedforward neural networks due to their dynamical nature. However, they have received less attention because training algorithms/architectures have not been well developed. In this study, a recursive least squares algorithm to train recurrent neural networks with an arbitrary number of hidden layers is developed. The training algorithm is developed as an extension of the standard recursive estimation problem. Simulated results obtained for identification of the dynamics of a nonlinear dynamical system show promising results.

Identification Of Cutting Force In End Milling Operations Using Recurrent Neural Networks, Q. Xu, K. Krishnamurthy, Bruce M. Mcmillin, Wen Feng Lu

Mechanical and Aerospace Engineering Faculty Research & Creative Works

The problem of identifying the cutting force in end milling operations is considered in this study. Recurrent neural networks are used here and are trained using a recursive least squares training algorithm. Training results for data obtained from a SAJO 3-axis vertical milling machine for steady slot cuts are presented. The results show that a recurrent neural network can learn the functional relationship between the feed rate and steady-state average resultant cutting force very well. Furthermore, results for the Mackey-Glass time series prediction problem are presented to illustrate the faster learning capability of the neural network scheme presented here