Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 2 of 2
Full-Text Articles in Entire DC Network
Novel Architectures And Optimization Algorithms For Training Neural Networks And Applications, Vasily I. Zadorozhnyy
Novel Architectures And Optimization Algorithms For Training Neural Networks And Applications, Vasily I. Zadorozhnyy
Theses and Dissertations--Mathematics
The two main areas of Deep Learning are Unsupervised and Supervised Learning. Unsupervised Learning studies a class of data processing problems in which only descriptions of objects are known, without label information. Generative Adversarial Networks (GANs) have become among the most widely used unsupervised neural net models. GAN combines two neural nets, generative and discriminative, that work simultaneously. We introduce a new family of discriminator loss functions that adopts a weighted sum of real and fake parts, which we call adaptive weighted loss functions. Using the gradient information, we can adaptively choose weights to train a discriminator in the direction …
Unitary And Symmetric Structure In Deep Neural Networks, Kehelwala Dewage Gayan Maduranga
Unitary And Symmetric Structure In Deep Neural Networks, Kehelwala Dewage Gayan Maduranga
Theses and Dissertations--Mathematics
Recurrent neural networks (RNNs) have been successfully used on a wide range of sequential data problems. A well-known difficulty in using RNNs is the vanishing or exploding gradient problem. Recently, there have been several different RNN architectures that try to mitigate this issue by maintaining an orthogonal or unitary recurrent weight matrix. One such architecture is the scaled Cayley orthogonal recurrent neural network (scoRNN), which parameterizes the orthogonal recurrent weight matrix through a scaled Cayley transform. This parametrization contains a diagonal scaling matrix consisting of positive or negative one entries that can not be optimized by gradient descent. Thus the …