Open Access. Powered by Scholars. Published by Universities.®
Physical Sciences and Mathematics Commons™
Open Access. Powered by Scholars. Published by Universities.®
- Keyword
-
- Deep Learning (2)
- Machine learning (2)
- Recurrent Neural Networks (2)
- Adaptively Weighted Discriminator Generative Adversarial Network (1)
- Assorted-Time Normalization (1)
-
- Batch Normalization (1)
- Convolutional Neural Networks (1)
- Deep learning (1)
- Exploding Gradients (1)
- Frobenius alphabets (1)
- Inverse-free preconditioned Krylov sub-space method (1)
- Isometries of codes (1)
- LU-LC conjecture (1)
- MacWillimas Extension Theorem (1)
- Machine Learning (1)
- Neural Networks (1)
- Neural networks (1)
- Orthogonal Gated Recurrent Unit with Neumann-Cayley Transformation (1)
- Protein Contact Map (1)
- Quantum stabilizer codes (1)
- RNA Secondary Structure (1)
- RNA secondary structure (1)
- Recurrent neural networks (1)
- Secondary structure inference (1)
- Self-Correcting Optimization (1)
- Singular value decomposition (1)
- Subspace clustering (1)
- Symmetric Convolutional Neural Network (1)
- Symmetrized CNN Architecture (1)
- Vanishing Gradients (1)
Articles 1 - 6 of 6
Full-Text Articles in Physical Sciences and Mathematics
Novel Architectures And Optimization Algorithms For Training Neural Networks And Applications, Vasily I. Zadorozhnyy
Novel Architectures And Optimization Algorithms For Training Neural Networks And Applications, Vasily I. Zadorozhnyy
Theses and Dissertations--Mathematics
The two main areas of Deep Learning are Unsupervised and Supervised Learning. Unsupervised Learning studies a class of data processing problems in which only descriptions of objects are known, without label information. Generative Adversarial Networks (GANs) have become among the most widely used unsupervised neural net models. GAN combines two neural nets, generative and discriminative, that work simultaneously. We introduce a new family of discriminator loss functions that adopts a weighted sum of real and fake parts, which we call adaptive weighted loss functions. Using the gradient information, we can adaptively choose weights to train a discriminator in the direction …
Unitary And Symmetric Structure In Deep Neural Networks, Kehelwala Dewage Gayan Maduranga
Unitary And Symmetric Structure In Deep Neural Networks, Kehelwala Dewage Gayan Maduranga
Theses and Dissertations--Mathematics
Recurrent neural networks (RNNs) have been successfully used on a wide range of sequential data problems. A well-known difficulty in using RNNs is the vanishing or exploding gradient problem. Recently, there have been several different RNN architectures that try to mitigate this issue by maintaining an orthogonal or unitary recurrent weight matrix. One such architecture is the scaled Cayley orthogonal recurrent neural network (scoRNN), which parameterizes the orthogonal recurrent weight matrix through a scaled Cayley transform. This parametrization contains a diagonal scaling matrix consisting of positive or negative one entries that can not be optimized by gradient descent. Thus the …
Orthogonal Recurrent Neural Networks And Batch Normalization In Deep Neural Networks, Kyle Eric Helfrich
Orthogonal Recurrent Neural Networks And Batch Normalization In Deep Neural Networks, Kyle Eric Helfrich
Theses and Dissertations--Mathematics
Despite the recent success of various machine learning techniques, there are still numerous obstacles that must be overcome. One obstacle is known as the vanishing/exploding gradient problem. This problem refers to gradients that either become zero or unbounded. This is a well known problem that commonly occurs in Recurrent Neural Networks (RNNs). In this work we describe how this problem can be mitigated, establish three different architectures that are designed to avoid this issue, and derive update schemes for each architecture. Another portion of this work focuses on the often used technique of batch normalization. Although found to be successful …
Equivalence Of Classical And Quantum Codes, Tefjol Pllaha
Equivalence Of Classical And Quantum Codes, Tefjol Pllaha
Theses and Dissertations--Mathematics
In classical and quantum information theory there are different types of error-correcting codes being used. We study the equivalence of codes via a classification of their isometries. The isometries of various codes over Frobenius alphabets endowed with various weights typically have a rich and predictable structure. On the other hand, when the alphabet is not Frobenius the isometry group behaves unpredictably. We use character theory to develop a duality theory of partitions over Frobenius bimodules, which is then used to study the equivalence of codes. We also consider instances of codes over non-Frobenius alphabets and establish their isometry groups. Secondly, …
Recurrent Neural Networks And Their Applications To Rna Secondary Structure Inference, Devin Willmott
Recurrent Neural Networks And Their Applications To Rna Secondary Structure Inference, Devin Willmott
Theses and Dissertations--Mathematics
Recurrent neural networks (RNNs) are state of the art sequential machine learning tools, but have difficulty learning sequences with long-range dependencies due to the exponential growth or decay of gradients backpropagated through the RNN. Some methods overcome this problem by modifying the standard RNN architecure to force the recurrent weight matrix W to remain orthogonal throughout training. The first half of this thesis presents a novel orthogonal RNN architecture that enforces orthogonality of W by parametrizing with a skew-symmetric matrix via the Cayley transform. We present rules for backpropagation through the Cayley transform, show how to deal with the Cayley …
Singular Value Computation And Subspace Clustering, Qiao Liang
Singular Value Computation And Subspace Clustering, Qiao Liang
Theses and Dissertations--Mathematics
In this dissertation we discuss two problems. In the first part, we consider the problem of computing a few extreme eigenvalues of a symmetric definite generalized eigenvalue problem or a few extreme singular values of a large and sparse matrix. The standard method of choice of computing a few extreme eigenvalues of a large symmetric matrix is the Lanczos or the implicitly restarted Lanczos method. These methods usually employ a shift-and-invert transformation to accelerate the speed of convergence, which is not practical for truly large problems. With this in mind, Golub and Ye proposes an inverse-free preconditioned Krylov subspace method, …