可数无限集上Shannon熵的有效Bernstein型界
An Effective Bernstein-type Bound on Shannon Entropy over Countably Infinite Alphabets
摘要:
我们证明了一个Bernstein型界之间的差异的平均值的负对数似然独立的离散随机变量和香农熵,都定义在一个可数无限的字母表。这一结果适用于尾部比离散幂律分布轻或与离散幂律分布同阶的离散随机变量。大多数常用的离散分布,如泊松分布,负二项分布和幂律分布本身都属于这一类。该界是有效的,因为我们提供了一种方法来计算其中的常数。
英文摘要:
We prove a Bernstein-type bound for the difference between the average of negative log-likelihoods of independent discrete random variables and the Shannon entropy, both defined on a countably infinite alphabet. The result holds for the class of discrete random variables with tails lighter than or on the same order of a discrete power-law distribution. Most commonly-used discrete distributions such as the Poisson distribution, the negative binomial distribution, and the power-law distribution itself belong to this class. The bound is effective in the sense that we provide a method to compute the constants in it.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。