Open Access. Powered by Scholars. Published by Universities.®
- Discipline
- Institution
- Publication
- Publication Type
Articles 1 - 4 of 4
Full-Text Articles in Other Applied Mathematics
Novel Architectures And Optimization Algorithms For Training Neural Networks And Applications, Vasily I. Zadorozhnyy
Novel Architectures And Optimization Algorithms For Training Neural Networks And Applications, Vasily I. Zadorozhnyy
Theses and Dissertations--Mathematics
The two main areas of Deep Learning are Unsupervised and Supervised Learning. Unsupervised Learning studies a class of data processing problems in which only descriptions of objects are known, without label information. Generative Adversarial Networks (GANs) have become among the most widely used unsupervised neural net models. GAN combines two neural nets, generative and discriminative, that work simultaneously. We introduce a new family of discriminator loss functions that adopts a weighted sum of real and fake parts, which we call adaptive weighted loss functions. Using the gradient information, we can adaptively choose weights to train a discriminator in the direction …
Quadratic Neural Network Architecture As Evaluated Relative To Conventional Neural Network Architecture, Reid Taylor
Quadratic Neural Network Architecture As Evaluated Relative To Conventional Neural Network Architecture, Reid Taylor
Senior Theses
Current work in the field of deep learning and neural networks revolves around several variations of the same mathematical model for associative learning. These variations, while significant and exceptionally applicable in the real world, fail to push the limits of modern computational prowess. This research does just that: by leveraging high order tensors in place of 2nd order tensors, quadratic neural networks can be developed and can allow for substantially more complex machine learning models which allow for self-interactions of collected and analyzed data. This research shows the theorization and development of mathematical model necessary for such an idea to …
Scaling Up Exact Neural Network Compression By Relu Stability, Thiago Serra, Xin Yu, Abhinav Kumar, Srikumar Ramalingam
Scaling Up Exact Neural Network Compression By Relu Stability, Thiago Serra, Xin Yu, Abhinav Kumar, Srikumar Ramalingam
Faculty Conference Papers and Presentations
We can compress a rectifier network while exactly preserving its underlying functionality with respect to a given input domain if some of its neurons are stable. However, current approaches to determine the stability of neurons with Rectified Linear Unit (ReLU) activations require solving or finding a good approximation to multiple discrete optimization problems. In this work, we introduce an algorithm based on solving a single optimization problem to identify all stable neurons. Our approach is on median 183 times faster than the state-of-art method on CIFAR-10, which allows us to explore exact compression on deeper (5 x 100) and wider …
Neural Machine Translation, Quinn M. Lanners, Thomas Laurent
Neural Machine Translation, Quinn M. Lanners, Thomas Laurent
Honors Thesis
Neural Machine Translation is the primary algorithm used in industry to perform machine translation. This state-of-the-art algorithm is an application of deep learning in which massive datasets of translated sentences are used to train a model capable of translating between any two languages. The architecture behind neural machine translation is composed of two recurrent neural networks used together in tandem to create an Encoder Decoder structure. Attention mechanisms have recently been developed to further increase the accuracy of these models. In this senior thesis, the various parts of Neural Machine Translation are explored towards the eventual creation of a tutorial …