Physical Sciences and Mathematics Commons^™

Learning Mixture Models With The Regularized Latent Maximum Entropy Principle, Shaojun Wang, Dale Schuurmans, Fuchun Peng, Yunxin Zhao

Kno.e.sis Publications

This paper presents a new approach to estimating mixture models based on a recent inference principle we have proposed: the latent maximum entropy principle (LME). LME is different from Jaynes' maximum entropy principle, standard maximum likelihood, and maximum a posteriori probability estimation. We demonstrate the LME principle by deriving new algorithms for mixture model estimation, and show how robust new variants of the expectation maximization (EM) algorithm can be developed. We show that a regularized version of LME (RLME), is effective at estimating mixture models. It generally yields better results than plain LME, which in turn is often better than …