非参数多元熵估计的最优分割
Optimal Partitions for Nonparametric Multivariate Entropy Estimation
摘要:
多元经验概率分布的有效和准确的估计是计算信息理论的措施,如互信息和转移熵的基础。常见的技术包括直方图估计的变化,虽然计算效率高,但通常无法精确捕获具有高相关性,峰度或精细子结构的样本的概率密度,特别是当样本量很小时。自适应分区,其根据样本进行调整,可以减少直方图本身的几何形状所带来的偏差,但这些通常集中在分区的位置,规模和粒度上,其效果对于高度相关的分布是有限的。在本文中,我重新制定的微分熵估计的特殊情况下,等概率直方图,使用k-d树划分的样本空间成箱的等概率质量。通过这样做,我暴露了一个隐式的旋转方向参数,这是次优指定在典型的边缘对齐。我建议,最佳的方向最大限度地减少了箱体积的方差,并证明了改进的熵估计可以通过旋转对齐分区的样本分布相应地获得。观察到这种最佳分区比现有技术更准确地估计具有已知理论值的相关双变量高斯分布的熵,在不同的样本大小(99%CI)。
英文摘要:
Efficient and accurate estimation of multivariate empirical probability distributions is fundamental to the calculation of information-theoretic measures such as mutual information and transfer entropy. Common techniques include variations on histogram estimation which, whilst computationally efficient, are often unable to precisely capture the probability density of samples with high correlation, kurtosis or fine substructure, especially when sample sizes are small. Adaptive partitions, which adjust heuristically to the sample, can reduce the bias imparted from the geometry of the histogram itself, but these have commonly focused on the location, scale and granularity of the partition, the effects of which are limited for highly correlated distributions. In this paper, I reformulate the differential entropy estimator for the special case of an equiprobable histogram, using a k-d tree to partition the sample space into bins of equal probability mass. By doing so, I expose an implicit rotational orientation parameter, which is conjectured to be suboptimally specified in the typical marginal alignment. I propose that the optimal orientation minimises the variance of the bin volumes, and demonstrate that improved entropy estimates can be obtained by rotationally aligning the partition to the sample distribution accordingly. Such optimal partitions are observed to be more accurate than existing techniques in estimating entropies of correlated bivariate Gaussian distributions with known theoretical values, across varying sample sizes (99% CI).
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。