论热力学与推理的对应
On the correspondence between thermodynamics and inference
摘要:
我们扩展贝叶斯统计和统计物理学之间的自然类比,其中样本量对应于逆温度。这个类比激发了两个新统计量的定义:学习能力和吉布斯熵。通过对热物理学中热容对应的学习能力的分析,对学习机制有了新的认识,并解释了为什么某些模型具有极高的学习性能。我们探讨了一些例子中的学习能力的属性,包括一个草率的模型。接下来,我们建议吉布斯熵提供了一个自然的设备计数可区分的分布在贝叶斯推理的上下文中。我们使用这种设备来定义一个广义的无差异原则(GPI),其中每个可区分的模型被分配相等的先验概率。这一原则的结果在一个新的解决方案,长期存在的问题,贝叶斯推理:定义一个客观的或无信息的先验。这种新方法的一个关键特点是,它可以应用于分析的模型维数是未知的,并规避贝叶斯推理中的高维模型的自动拒绝。
英文摘要:
We expand upon a natural analogy between Bayesian statistics and statistical physics in which sample size corresponds to inverse temperature. This analogy motivates the definition of two novel statistical quantities: a learning capacity and a Gibbs entropy. The analysis of the learning capacity, corresponding to the heat capacity in thermal physics, leads to new insight into the mechanism of learning and explains why some models have anomalously-high learning performance. We explore the properties of the learning capacity in a number of examples, including a sloppy model. Next, we propose that the Gibbs entropy provides a natural device for counting distinguishable distributions in the context of Bayesian inference. We use this device to define a generalized principle of indifference (GPI) in which every distinguishable model is assigned equal a priori probability. This principle results in a new solution to a long-standing problem in Bayesian inference: the definition of an objective or uninformative prior. A key characteristic of this new approach is that it can be applied to analyses where the model dimension is unknown and circumvents the automatic rejection of higher-dimensional models in Bayesian inference.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。