Physical Sciences and Mathematics Commons^™

Dedicated Hardware For Machine/Deep Learning: Domain Specific Architectures, Angel Izael Solis

Open Access Theses & Dissertations

Artificial intelligence has come a very long way from being a mere spectacle on the silver screen in the 1920s [Hml18]. As artificial intelligence continues to evolve, and we begin to develop more sophisticated Artificial Neural Networks, the need for specialized and more efficient machines (less computational strain while maintaining the same performance results) becomes increasingly evident. Though these “new” techniques, such as Multilayer Perceptron’s, Convolutional Neural Networks and Recurrent Neural Networks, may seem as if they are on the cutting edge of technology, many of these ideas are over 60 years old! However, many of these earlier models, at …

A Novel Set Of Weight Initialization Techniques For Deep Learning Architectures, Diego Aguirre

Open Access Theses & Dissertations

The importance of weight initialization when building a deep learning model is often underappreciated. Even though it is usually seen as a minor detail in the model creation cycle, this process has shown to have a strong impact on the training time of a network and the quality of the resulting model. In fact, the implications of choosing a poor initialization scheme range from leading to the creation of a poorly performing model to preventing optimization techniques (like stochastic gradient descent) from converging.

In this work, we introduce and evaluate a set of novel weight initialization techniques for deep learning …