Open Access. Powered by Scholars. Published by Universities.®
Physical Sciences and Mathematics Commons™
Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 2 of 2
Full-Text Articles in Physical Sciences and Mathematics
Speed Training: Improving The Rate Of Backpropagation Learning Through Stochastic Sample Presentation, Timothy L. Andersen, Tony R. Martinez, Michael E. Rimer
Speed Training: Improving The Rate Of Backpropagation Learning Through Stochastic Sample Presentation, Timothy L. Andersen, Tony R. Martinez, Michael E. Rimer
Faculty Publications
Artificial neural networks provide an effective empirical predictive model for pattern classification. However, using complex neural networks to learn very large training sets is often problematic, imposing prohibitive time constraints on the training process. We present four practical methods for dramatically decreasing training time through dynamic stochastic sample presentation, a technique we call speed training. These methods are shown to be robust to retaining generalization accuracy over a diverse collection of real world data sets. In particular, the SET technique achieves a training speedup of 4278% on a large OCR database with no detectable loss in generalization.
The Need For Small Learning Rates On Large Problems, Tony R. Martinez, D. Randall Wilson
The Need For Small Learning Rates On Large Problems, Tony R. Martinez, D. Randall Wilson
Faculty Publications
In gradient descent learning algorithms such as error backpropagation, the learning rate parameter can have a significant effect on generalization accuracy. In particular, decreasing the learning rate below that which yields the fastest convergence can significantly improve generalization accuracy, especially on large, complex problems. The learning rate also directly affects training speed, but not necessarily in the way that many people expect. Many neural network practitioners currently attempt to use the largest learning rate that still allows for convergence, in order to improve training speed. However, a learning rate that is too large can be as slow as a learning …