arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.10491hep-thcond-mat.dis-nncs.AI
收藏

贝叶斯重整化

Bayesian Renormalization

Queen Mary University of London(伦敦玛丽女王大学);University of Illinois(伊利诺伊大学)

David S. Berman, Marc S. Klinger, Alexander G. Stapleton

AI总结:

提出贝叶斯重整化,利用Fisher度量定义信息论RG尺度,在无物理尺度时适用,并设计显式方案验证其按信息重要性组织神经网络参数。

摘要:

在本文中,我们提出了一种完全基于信息论的重整化方法,其灵感来源于贝叶斯统计推断,我们称之为贝叶斯重整化。贝叶斯重整化的主要洞见在于,Fisher度量定义了一个关联长度,该长度扮演了涌现重整化群(RG)尺度的角色,用于量化概率分布空间中邻近点之间的可区分性。这个RG尺度可以被解释为在统计推断实验中对给定系统所能做出的最大唯一观测数量的代理。贝叶斯重整化方案的作用随后是为给定系统准备一个有效模型,其精度受上述尺度限制。在将贝叶斯重整化应用于物理系统时,涌现的信息论尺度自然地被等同于当前实验设备所能探测的最大能量,因此贝叶斯重整化与普通重整化一致。然而,贝叶斯重整化具有足够的通用性,即使在缺乏直接物理尺度的情况下也能适用,从而为数据科学背景下的重整化提供了一种理想方法。为此,我们深入探讨了贝叶斯重整化方案与现有数据压缩和数据生成方法(如信息瓶颈和扩散学习范式)之间的关系。最后,我们受量子场论中Wilson动量壳重整化方案的启发,设计了一种显式的贝叶斯重整化形式。我们将这种贝叶斯重整化方案应用于一个简单的神经网络,并验证了它如何根据信息论重要性的层级来组织模型参数。

英文摘要:

In this note we present a fully information theoretic approach to renormalization inspired by Bayesian statistical inference, which we refer to as Bayesian Renormalization. The main insight of Bayesian Renormalization is that the Fisher metric defines a correlation length that plays the role of an emergent RG scale quantifying the distinguishability between nearby points in the space of probability distributions. This RG scale can be interpreted as a proxy for the maximum number of unique observations that can be made about a given system during a statistical inference experiment. The role of the Bayesian Renormalization scheme is subsequently to prepare an effective model for a given system up to a precision which is bounded by the aforementioned scale. In applications of Bayesian Renormalization to physical systems, the emergent information theoretic scale is naturally identified with the maximum energy that can be probed by current experimental apparatus, and thus Bayesian Renormalization coincides with ordinary renormalization. However, Bayesian Renormalization is sufficiently general to apply even in circumstances in which an immediate physical scale is absent, and thus provides an ideal approach to renormalization in data science contexts. To this end, we provide insight into how the Bayesian Renormalization scheme relates to existing methods for data compression and data generation such as the information bottleneck and the diffusion learning paradigm. We conclude by designing an explicit form of Bayesian Renormalization inspired by Wilson's momentum shell renormalization scheme in Quantum Field Theory. We apply this Bayesian Renormalization scheme to a simple Neural Network and verify the sense in which it organizes the parameters of the model according to a hierarchy of information theoretic importance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑