arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1107.6004cs.ITmath.ITphysics.data-an
收藏

线性约束下熵集中的显式界

Explicit Bounds for Entropy Concentration under Linear Constraints

Kostas N. Oikonomou, Peter D. Grunwald

摘要:

考虑$n$结果的所有序列的集合,每个序列采用满足多个线性约束的$m$值之一。如果$m$固定而$n$增加,则满足约束的大多数序列导致其熵接近满足约束的最大熵向量的熵的频率向量。这种著名的“熵集中”现象是最大熵方法的基础。集中现象的现有证明是基于极限或渐近性,并且不切实际地假设约束精确地保持,在原则上支持最大熵推论而不是在实践中。我们目前,第一次,非渐近,明确的下限$n$的浓度结果的一些变种,以保持任何规定的精度,与约束保持任何指定的公差,考虑到这样一个事实,即分配的离散单元可以满足约束只有近似。与先前的结果不同,我们测量浓度不是通过偏离最大熵值,而是通过$\ell_1$和$\ell_2$与最大熵实现频率矢量的距离。我们的一个结果独立于字母表大小$m$,是基于一种新的证明技术,使用多维贝里-埃辛定理。我们用各种详细的例子说明和比较我们的结果。

英文摘要:

Consider the set of all sequences of $n$ outcomes, each taking one of $m$ values, that satisfy a number of linear constraints. If $m$ is fixed while $n$ increases, most sequences that satisfy the constraints result in frequency vectors whose entropy approaches that of the maximum entropy vector satisfying the constraints. This well-known "entropy concentration" phenomenon underlies the maximum entropy method. Existing proofs of the concentration phenomenon are based on limits or asymptotics and unrealistically assume that constraints hold precisely, supporting maximum entropy inference more in principle than in practice. We present, for the first time, non-asymptotic, explicit lower bounds on $n$ for a number of variants of the concentration result to hold to any prescribed accuracies, with the constraints holding up to any specified tolerance, taking into account the fact that allocations of discrete units can satisfy constraints only approximately. Again unlike earlier results, we measure concentration not by deviation from the maximum entropy value, but by the $\ell_1$ and $\ell_2$ distances from the maximum entropy-achieving frequency vector. One of our results holds independently of the alphabet size $m$ and is based on a novel proof technique using the multi-dimensional Berry-Esseen theorem. We illustrate and compare our results using various detailed examples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑