Engineering Commons^™

Rnn Language Model With Word Clustering And Class-Based Output Layer, Yongzhe Shi, Wei-Qiang Zhang, Jia Liu, Michael T. Johnson

Electrical and Computer Engineering Faculty Research and Publications

The recurrent neural network language model (RNNLM) has shown significant promise for statistical language modeling. In this work, a new class-based output layer method is introduced to further improve the RNNLM. In this method, word class information is incorporated into the output layer by utilizing the Brown clustering algorithm to estimate a class-based language model. Experimental results show that the new output layer with word clustering not only improves the convergence obviously but also reduces the perplexity and word error rate in large vocabulary continuous speech recognition.