奇异贝叶斯模型中的热力学响应函数
Thermodynamic Response Functions in Singular Bayesian Models
AI总结：
本文提出通过热力学响应理论解释奇异贝叶斯模型中的复杂性、预测波动和结构重组。
摘要：
奇异统计模型-包括混合模型、矩阵分解和神经网络-由于参数非可识别性和退化费舍尔几何而违反常规渐近性。尽管奇异学习理论通过诸如真实对数 canonical 阈值和奇异波动这样的不变量来表征边缘似然行为，但这些量仍然难以操作性地解释。同时，广泛使用的标准如WAIC和WBIC似乎与底层的奇异几何脱节。我们证明后验温度化诱导了后验分布的一参数变形，其相关的可观测量生成一系列热力学响应函数的层次结构。一个通用的协方差恒等式将温度化期望的导数与后验波动联系起来，将WAIC、WBIC和奇异波动置于统一的响应框架中。在此框架内，经典量从奇异学习理论获得自然的热力学解释：RLCT支配主导自由能斜率，奇异波动对应于温度化自由能的曲率，而WAIC测量预测波动。我们正式化一个可观测量代数，消去非可识别方向，允许在奇异模型中构造结构上意义的序参数。在经典的奇异示例中-包括对称高斯混合、降维秩回归和过参数化神经网络-我们经验性地展示了在温度化下的相变行为。序参数坍塌，易感性峰值，复杂性度量与后验几何的结构重组一致。我们的结果表明，热力学响应理论为解释奇异贝叶斯学习中的复杂性、预测波动和结构重组提供了自然的组织框架。
英文摘要：
Singular statistical models-including mixtures, matrix factorization, and neural networks-violate regular asymptotics due to parameter non-identifiability and degenerate Fisher geometry. Although singular learning theory characterizes marginal likelihood behavior through invariants such as the real log canonical threshold and singular fluctuation, these quantities remain difficult to interpret operationally. At the same time, widely used criteria such as WAIC and WBIC appear disconnected from underlying singular geometry. We show that posterior tempering induces a one-parameter deformation of the posterior distribution whose associated observables generate a hierarchy of thermodynamic response functions. A universal covariance identity links derivatives of tempered expectations to posterior fluctuations, placing WAIC, WBIC, and singular fluctuation within a unified response framework. Within this framework, classical quantities from singular learning theory acquire natural thermodynamic interpretations: RLCT governs the leading free-energy slope, singular fluctuation corresponds to curvature of the tempered free energy, and WAIC measures predictive fluctuation. We formalize an observable algebra that quotients out non-identifiable directions, allowing structurally meaningful order parameters to be constructed in singular models. Across canonical singular examples-including symmetric Gaussian mixtures, reduced-rank regression, and overparameterized neural networks-we empirically demonstrate phase-transition-like behavior under tempering. Order parameters collapse, susceptibilities peak, and complexity measures align with structural reorganization in posterior geometry. Our results suggest that thermodynamic response theory provides a natural organizing framework for interpreting complexity, predictive variability, and structural reorganization in singular Bayesian learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
