Numerical Analysis and Computation Commons^™

Orthogonal Recurrent Neural Networks And Batch Normalization In Deep Neural Networks, Kyle Eric Helfrich

Theses and Dissertations--Mathematics

Despite the recent success of various machine learning techniques, there are still numerous obstacles that must be overcome. One obstacle is known as the vanishing/exploding gradient problem. This problem refers to gradients that either become zero or unbounded. This is a well known problem that commonly occurs in Recurrent Neural Networks (RNNs). In this work we describe how this problem can be mitigated, establish three different architectures that are designed to avoid this issue, and derive update schemes for each architecture. Another portion of this work focuses on the often used technique of batch normalization. Although found to be successful …

Unitary And Symmetric Structure In Deep Neural Networks, Kehelwala Dewage Gayan Maduranga

Theses and Dissertations--Mathematics

Recurrent neural networks (RNNs) have been successfully used on a wide range of sequential data problems. A well-known difficulty in using RNNs is the vanishing or exploding gradient problem. Recently, there have been several different RNN architectures that try to mitigate this issue by maintaining an orthogonal or unitary recurrent weight matrix. One such architecture is the scaled Cayley orthogonal recurrent neural network (scoRNN), which parameterizes the orthogonal recurrent weight matrix through a scaled Cayley transform. This parametrization contains a diagonal scaling matrix consisting of positive or negative one entries that can not be optimized by gradient descent. Thus the …

On The Role Of Ill-Conditioning: Biharmonic Eigenvalue Problem And Multigrid Algorithms, Kasey Bray

Theses and Dissertations--Mathematics

Very fine discretizations of differential operators often lead to large, sparse matrices A, where the condition number of A is large. Such ill-conditioning has well known effects on both solving linear systems and eigenvalue computations, and, in general, computing solutions with relative accuracy independent of the condition number is highly desirable. This dissertation is divided into two parts.

In the first part, we discuss a method of preconditioning, developed by Ye, which allows solutions of Ax=b to be computed accurately. This, in turn, allows for accurate eigenvalue computations. We then use this method to develop discretizations that yield accurate computations …

The Krylov Subspace Methods For The Computation Of Matrix Exponentials, Hao Wang

Theses and Dissertations--Mathematics

The problem of computing the matrix exponential e^tA arises in many theoretical and practical problems. Many methods have been developed to accurately and efficiently compute this matrix function or its product with a vector, i.e., e^tAv. In the past few decades, with the increasing need of the computation for large sparse matrices, iterative methods such as the Krylov subspace methods have proved to be a powerful class of methods in dealing with many linear algebra problems. The Krylov subspace methods have been introduced for computing matrix exponentials by Gallopoulos and Saad, and the corresponding error bounds …

Singular Value Computation And Subspace Clustering, Qiao Liang

Theses and Dissertations--Mathematics

In this dissertation we discuss two problems. In the first part, we consider the problem of computing a few extreme eigenvalues of a symmetric definite generalized eigenvalue problem or a few extreme singular values of a large and sparse matrix. The standard method of choice of computing a few extreme eigenvalues of a large symmetric matrix is the Lanczos or the implicitly restarted Lanczos method. These methods usually employ a shift-and-invert transformation to accelerate the speed of convergence, which is not practical for truly large problems. With this in mind, Golub and Ye proposes an inverse-free preconditioned Krylov subspace method, …

A Posteriori Error Estimates For Surface Finite Element Methods, Fernando F. Camacho

Theses and Dissertations--Mathematics

Problems involving the solution of partial differential equations over surfaces appear in many engineering and scientific applications. Some of those applications include crystal growth, fluid mechanics and computer graphics. Many times analytic solutions to such problems are not available. Numerical algorithms, such as Finite Element Methods, are used in practice to find approximate solutions in those cases.

In this work we present L2 and pointwise a posteriori error estimates for Adaptive Surface Finite Elements solving the Laplace-Beltrami equation −△_Γ u = f . The two sources of errors for Surface Finite Elements are a Galerkin error, and a …