arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 0809.1017cs.ITcs.LGmath.ITmath.STstat.MEstat.TH
收藏

熵集中与经验编码博弈

Entropy Concentration and the Empirical Coding Game

CWI(CWI(荷兰数学和计算机科学研究学会));EURANDOM(EURANDOM(埃因霍温概率与统计研究所))

Peter Grunwald

摘要:

我们给出了最大熵/最小相对熵推理的特征,通过提供两个“强熵集中”定理。这些定理统一和推广了Jaynes的“集中现象”和Van Campenhout和Cover的“条件极限定理”。这些定理精确地描述了在给定约束条件下的先验分布Q和分布P在什么意义上最小化相对熵D(P|| Q)在满足约束的所有分布上,彼此“接近”。然后,我们应用我们的定理,建立熵浓度和最大熵推理由于托普索和其他博弈论表征之间的关系。

英文摘要:

We give a characterization of Maximum Entropy/Minimum Relative Entropy inference by providing two `strong entropy concentration' theorems. These theorems unify and generalize Jaynes' `concentration phenomenon' and Van Campenhout and Cover's `conditional limit theorem'. The theorems characterize exactly in what sense a prior distribution Q conditioned on a given constraint, and the distribution P, minimizing the relative entropy D(P ||Q) over all distributions satisfying the constraint, are `close' to each other. We then apply our theorems to establish the relationship between entropy concentration and a game-theoretic characterization of Maximum Entropy Inference due to Topsoe and others.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑