arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 0902.3038cond-mat.stat-mech
收藏

组合熵与统计

Combinatorial Entropies and Statistics

Robert K. Niven

摘要:

我们研究熵或交叉熵函数$H \propto \ln \mathbb{W}$或$D \propto - \ln \mathbb{P}$的{组合}或{概率}定义(“玻尔兹曼原理”),其中$\mathbb{W}$是统计权重,$\mathbb{P}$是系统给定实现的概率。因此,受任何约束的$H$或$D$的极值选择“最可能”(MaxProb)实现。如果系统是多项式系统,$D$(对于实体数$N \back \to \back \infty$)渐近收敛于Kullback-Leibler交叉熵$D_{KL}$ ;对于系统中的等概率范畴,$H$收敛于香农熵$H_{Sh}$。然而,在许多情况下,$\mathbb{W}$或$\mathbb{P}$不是多项式和/或不满足渐近极限。这些系统不能用$D_{KL}$或$H_{Sh}$进行有意义的分析,但可以直接用MaxProb进行分析。本研究回顾了几个例子,包括(a)非渐近系统;(b)具有不可区分实体的系统(量子统计);(c)具有不可区分类别的系统;(d)由urn模型表示的系统,如“既不独立也不同分布”(ninid)抽样;以及(e)以图形形式表示的系统,如决策树和网络。玻尔兹曼的熵的组合定义被证明是更重要的{“概率推理”}比信息论中使用的公理化定义。

英文摘要:

We examine the {combinatorial} or {probabilistic} definition ("Boltzmann's principle") of the entropy or cross-entropy function $H \propto \ln \mathbb{W}$ or $D \propto - \ln \mathbb{P}$, where $\mathbb{W}$ is the statistical weight and $\mathbb{P}$ the probability of a given realization of a system. Extremisation of $H$ or $D$, subject to any constraints, thus selects the "most probable" (MaxProb) realization. If the system is multinomial, $D$ converges asymptotically (for number of entities $N \back \to \back \infty$) to the Kullback-Leibler cross-entropy $D_{KL}$; for equiprobable categories in a system, $H$ converges to the Shannon entropy $H_{Sh}$. However, in many cases $\mathbb{W}$ or $\mathbb{P}$ is not multinomial and/or does not satisfy an asymptotic limit. Such systems cannot meaningfully be analysed with $D_{KL}$ or $H_{Sh}$, but can be analysed directly by MaxProb. This study reviews several examples, including (a) non-asymptotic systems; (b) systems with indistinguishable entities (quantum statistics); (c) systems with indistinguishable categories; (d) systems represented by urn models, such as "neither independent nor identically distributed" (ninid) sampling; and (e) systems representable in graphical form, such as decision trees and networks. Boltzmann's combinatorial definition of entropy is shown to be of greater importance for {"probabilistic inference"} than the axiomatic definition used in information theory.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑