arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 0908.3961stat.COmath.STstat.TH
收藏

一种简单的熵估计草图算法

A simple sketching algorithm for entropy estimation

Peter Clifford, Ioana Ada Cosma

摘要:

我们考虑的问题,近似的经验香农熵的高频数据流下放松严格的旋转门模型,当空间的限制,使精确的计算不可行。熵的等价度量是依赖于常数α的Renyi熵。这个数量可以有效地估计和无偏地从一个低维的概要称为阿尔法稳定的数据草图通过压缩计数的方法。香农熵的近似值可以通过取α足够接近1从雷尼熵得到。然而,缺乏关于α的参数校准的实用指南。我们避免了这个问题,表明在估计Renyi熵中使用的随机变量可以被转换为具有适当的分布限制,因为α接近1:最大偏斜,严格稳定分布与α = 1定义在整个实数线。我们提出了一个渐近无偏的对数均值估计的香农熵,索引常数zeta > 0,可以计算在一个单遍算法提供一个添加剂的近似。我们建议zeta = 1的对数均值估计,具有指数下降的尾界的错误概率,渐近相对效率为0.932,和接近最优的计算复杂性。

英文摘要:

We consider the problem of approximating the empirical Shannon entropy of a high-frequency data stream under the relaxed strict-turnstile model, when space limitations make exact computation infeasible. An equivalent measure of entropy is the Renyi entropy that depends on a constant alpha. This quantity can be estimated efficiently and unbiasedly from a low-dimensional synopsis called an alpha-stable data sketch via the method of compressed counting. An approximation to the Shannon entropy can be obtained from the Renyi entropy by taking alpha sufficiently close to 1. However, practical guidelines for parameter calibration with respect to alpha are lacking. We avoid this problem by showing that the random variables used in estimating the Renyi entropy can be transformed to have a proper distributional limit as alpha approaches 1: the maximally skewed, strictly stable distribution with alpha = 1 defined on the entire real line. We propose a family of asymptotically unbiased log-mean estimators of the Shannon entropy, indexed by a constant zeta > 0, that can be computed in a single-pass algorithm to provide an additive approximation. We recommend the log-mean estimator with zeta = 1 that has exponentially decreasing tail bounds on the error probability, asymptotic relative efficiency of 0.932, and near-optimal computational complexity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑