等概率随机样本一维熵的精确概率估计
Computing Accurate Probabilistic Estimates of One-D Entropy from Equiprobable Random Samples
摘要：
本文提出了一种简单的分位数间距（QS）方法，用于等概率随机样本一维熵的精确概率估计，并将其与常用的分位数计数（BC）方法进行了比较。与使用具有不同概率质量的等宽箱的BC相反，QS方法使用将生成概率密度函数（pdf）的数据的支持划分为等概率质量区间的分位数的估计。而BC需要优化调整bin宽度超参数，其值随样本量和pdf的形状而变化，QS需要指定要使用的分位数的数量。结果表明，对于所测试的分布类别，分位数间距的最佳数量是样本量的固定分数（根据经验确定为~0.25-0.35），并且该值对分布形式或样本量相对不敏感，因为不需要超参数调整，因此提供了优于BC的明显优势。Bootstrapping是用来近似的抽样变异分布所产生的熵估计，并准确地反映了真实的不确定性。对于研究的四种分布形式（高斯、对数正态、指数和双峰高斯混合），预期估计偏差小于1%，即使对于非常小的样本量，不确定性也相对较低。我们推测，估计分位数的位置，而不是bin概率，结果在更有效地利用数据中的信息，以近似的潜在形状的未知数据生成PDF。
英文摘要：
We develop a simple Quantile Spacing (QS) method for accurate probabilistic estimation of one-dimensional entropy from equiprobable random samples, and compare it with the popular Bin-Counting (BC) method. In contrast to BC, which uses equal-width bins with varying probability mass, the QS method uses estimates of the quantiles that divide the support of the data generating probability density function (pdf) into equal-probability-mass intervals. Whereas BC requires optimal tuning of a bin-width hyper-parameter whose value varies with sample size and shape of the pdf, QS requires specification of the number of quantiles to be used. Results indicate, for the class of distributions tested, that the optimal number of quantile-spacings is a fixed fraction of the sample size (empirically determined to be ~0.25-0.35), and that this value is relatively insensitive to distributional form or sample size, providing a clear advantage over BC since hyperparameter tuning is not required. Bootstrapping is used to approximate the sampling variability distribution of the resulting entropy estimate, and is shown to accurately reflect the true uncertainty. For the four distributional forms studied (Gaussian, Log-Normal, Exponential and Bimodal Gaussian Mixture), expected estimation bias is less than 1% and uncertainty is relatively low even for very small sample sizes. We speculate that estimating quantile locations, rather than bin-probabilities, results in more efficient use of the information in the data to approximate the underlying shape of an unknown data generating pdf.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
