Open Access. Powered by Scholars. Published by Universities.®
Physical Sciences and Mathematics Commons™
Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 2 of 2
Full-Text Articles in Physical Sciences and Mathematics
Lazy Training: Improving Backpropagation Learning Through Network Interaction, Timothy L. Andersen, Tony R. Martinez, Michael E. Rimer
Lazy Training: Improving Backpropagation Learning Through Network Interaction, Timothy L. Andersen, Tony R. Martinez, Michael E. Rimer
Faculty Publications
Backpropagation, similar to most high-order learning algorithms, is prone to overfitting. We address this issue by introducing interactive training (IT), a logical extension to backpropagation training that employs interaction among multiple networks. This method is based on the theory that centralized control is more effective for learning in deep problem spaces in a multi-agent paradigm. IT methods allow networks to work together to form more complex systems while not restraining their individual ability to specialize. Lazy training, an implementation of IT that minimizes misclassification error, is presented. Lazy training discourages overfitting and is conducive to higher accuracy in multiclass problems …
The Need For Small Learning Rates On Large Problems, Tony R. Martinez, D. Randall Wilson
The Need For Small Learning Rates On Large Problems, Tony R. Martinez, D. Randall Wilson
Faculty Publications
In gradient descent learning algorithms such as error backpropagation, the learning rate parameter can have a significant effect on generalization accuracy. In particular, decreasing the learning rate below that which yields the fastest convergence can significantly improve generalization accuracy, especially on large, complex problems. The learning rate also directly affects training speed, but not necessarily in the way that many people expect. Many neural network practitioners currently attempt to use the largest learning rate that still allows for convergence, in order to improve training speed. However, a learning rate that is too large can be as slow as a learning …