arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.09031cs.LG
收藏

球面玻尔兹曼机:能量基模型中学习与生成的可解理论

Spherical Boltzmann machines: a solvable theory of learning and generation in energy-based models

Laboratoire de Physique de l’École Normale Supérieure, PSL, CNRS UMR8023, Sorbonne Université(巴黎高等师范学院物理实验室,PSL,CNRS UMR8023,索邦大学);Institut de Physique Théorique, Université Paris-Saclay, CNRS UMR3681, CEA(巴黎-萨克雷大学理论物理研究所,CNRS UMR3681,CEA)

Thomas Tulinski, Simona Cocco, Rémi Monasson, Jorge Fernandez-De-Cossio-Diaz

AI总结:

本文研究了球面玻尔兹曼机在高维极限下的可解性,通过随机矩阵理论和动力学平均场理论,推导了训练动态方程,计算了贝叶斯证据,并揭示了训练和超参数变化过程中相变 cascade,连接到教师-学生场景中的采样时间生成现象。

摘要:

能量基模型(EBMs)是受统计物理启发的灵活生成架构,但其学习和生成性质仍不明确。本文分析了在高维极限下的可解 EBM:球面玻尔兹曼机(SBM)。结合随机矩阵理论和动力学平均场理论,我们:推导了描述 SBM 训练动态的精确方程;计算了贝叶斯证据,其在参数空间中作为配分函数,并编码训练模型的全局性质;并揭示了在训练过程中以及超参数变化时发生的相变 cascade,这些相变与耦合矩阵顶部模式与数据对齐和凝聚有关。我们将这些相变与教师-学生场景中的采样时间生成现象联系起来,包括:采样温度调节、正则化强度函数上的双下降、温后验效应,以及训练过程中非平衡效应导致的模型偏差。我们提供了数值证据,证明这些现象出现在标准生成架构中,而不仅限于 SBM。

英文摘要:

Energy-based models (EBMs) are flexible generative architectures inspired by statistical physics, but their learning and generative properties remain poorly understood. Here, we analyze a solvable EBM in the high-dimensional limit: the spherical Boltzmann machine (SBM). Combining tools from random matrix theory and dynamical mean-field theory, we: solve exact equations describing the training dynamics of the SBM; compute the Bayesian evidence, which acts as a partition function in parameter space and encodes global properties of the trained model; and uncover cascades of phase transitions that occur both during training and as a function of hyperparameters, related to successive alignment and condensation of the top modes of the coupling matrix to the data. We connect these transitions to sampling-time generative phenomena in a teacher-student scenario, including: sampling temperature tuning, double descent as a function of regularization strength, tempered posterior effects, and out-of-equilibrium effects during training that induce biases in the trained model. We provide numerical evidence demonstrating that all these phenomena appear in standard generative architectures, beyond the SBM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑