arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.13625cond-mat.stat-mechastro-ph.COcs.ITmath.IT
收藏

非高斯似然的配分函数方法:贝叶斯推断的信息论与状态变量

Partition function approach to non-Gaussian likelihoods: information theory and state variables for Bayesian inference

Rebecca Maria Kuntz, Heinrich von Campe, Tobias Röspel, Maximilian Philipp Herzog, Björn Malte Schäfer

AI总结:

本文基于统计力学与贝叶斯推断的相似性,构建了贝叶斯更新的连续热力学模型,将热力学和统计力学的形式体系引入统计推断,并利用宇宙学数值例子进行验证。

摘要:

熵等统计物理学概念的意义远超经典热力学。我们将统计力学中的配分与贝叶斯推断中的配分之间的相似性,解释为对Jaynes(1957)结果的阐述,他阐明了热力学本质上是一种信息论。在此框架下,每个采样过程都有对应的力学类比。因此,参数空间中采样器系综与从热力学平衡的力学系统中采样之间的界限是人为设定的。基于这一认识,我们构建了贝叶斯更新的连续建模,类似于热力学系综之间的转变。这引出了Jazinsky等式的信息论解释,将做功的消耗与数据通过似然产生的影响联系起来。我们提出了一种方法,从贝叶斯定律出发,将热力学(能量、功、热量)和统计力学(配分函数)的词汇与形式体系转移到统计推断中。讨论了不同类型的推断过程,并证明相对熵作为采样过程的分析公式,可从适当构建的配分中推导得出。最后,我们提出将有效维度作为系统复杂度的度量。提出了一个宇宙学的数值例子来说明这些结果。

英文摘要:

The significance of statistical physics concepts such as entropy extends far beyond classical thermodynamics. We interpret the similarity between partitions in statistical mechanics and partitions in Bayesian inference as an articulation of a result by Jaynes (1957), who clarified that thermodynamics is in essence a theory of information. In this, every sampling process has a mechanical analogue. Consequently, the divide between ensembles of samplers in parameter space and sampling from a mechanical system in thermodynamic equilibrium would be artificial. Based on this realisation, we construct a continuous modelling of a Bayes update akin to a transition between thermodynamic ensembles. This leads to an information theoretic interpretation of Jazinsky's equality, relating the expenditure of work to the influence of data via the likelihood. We propose one way to transfer the vocabulary and the formalism of thermodynamics (energy, work, heat) and statistical mechanics (partition functions) to statistical inference, starting from Bayes' law. Different kinds of inference processes are discussed and relative entropies are shown to follow from suitably constructed partitions as an analytical formulation of sampling processes. Lastly, we propose an effective dimension as a measure of system complexity. A numerical example from cosmology is put forward to illustrate these results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑