Open Access. Powered by Scholars. Published by Universities.®

Physical Sciences and Mathematics Commons

Open Access. Powered by Scholars. Published by Universities.®

Computer Sciences

Selected Works

Erik G Learned-Miller

Selected Works

2004

Articles 1 - 1 of 1

Full-Text Articles in Physical Sciences and Mathematics

Hyperspacings And The Estimation Of Information Theoretic Quantities, Erik G. Learned-Miller Dec 2003

Hyperspacings And The Estimation Of Information Theoretic Quantities, Erik G. Learned-Miller

Erik G Learned-Miller

The estimation of probability densities from data is widely used as an intermediate step in the estimation of entropy, Kullback-Leibler (KL) divergence, and mutual information, and for statistical tasks such as hypothesis testing. We propose an alternative to density estimation– partitioning a space into regions whose approximate probability mass is known–that can be used for the same purposes. We call these regions hyperspacings, a generalization of spacings in one dimension. After discussing one-dimensional spacings estimates of entropy and KL-divergence, we show how hyperspacings can be used to estimate these quantities (and mutual information) in higher dimensions. Our approach outperforms certain …