Special Functions Commons^™

Uniform Convergence Of Deep Neural Networks With Lipschitz Continuous Activation Functions And Variable Widths, Yuesheng Xu, Haizhang Zhang

Mathematics & Statistics Faculty Publications

We consider deep neural networks (DNNs) with a Lipschitz continuous activation function and with weight matrices of variable widths. We establish a uniform convergence analysis framework in which sufficient conditions on weight matrices and bias vectors together with the Lipschitz constant are provided to ensure uniform convergence of DNNs to a meaningful function as the number of their layers tends to infinity. In the framework, special results on uniform convergence of DNNs with a fixed width, bounded widths and unbounded widths are presented. In particular, as convolutional neural networks are special DNNs with weight matrices of increasing widths, we put …