发表机构
The University of Tokyo（东京大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对变分自编码器（VAE）的后验崩溃问题，构建残余谱涨落理论，通过数值实验验证潜在维度随谱边际逐步丢失的现象，为表征学习的稳定性提供谱级联解释。
AI 中文摘要
学习到的表征会逐步丢失潜在自由度，这表明存在一系列转变，但其潜在的稳定性原理仍不清楚。本文将变分自编码器（VAE）中按维度划分的后验崩溃问题，表述为部分崩溃状态附近的涨落理论。将负证据下界视为有效自由能，其二次展开定义了一种高斯理论，该理论的海森矩阵充当潜在涨落的质量矩阵。我们证明，崩溃方向构成一个不变涨落区，并根据条件残余算子推导其精确质量谱。当解码器方差降至残余谱上限以下时，局部重新激活方向会降低自由能，相等时标记为边际状态。该准则在线性高斯VAE极限下可恢复主成分阈值。沿连续连接分支反向观察，重新激活边界为连续崩溃提供了局部准则。数值延拓实验显示，在这些谱边际附近，潜在维度会逐步丢失。这些结果支持由存活表征未解释的残余信息所支配的谱级联解释。
英文摘要
Learned representations can lose latent degrees of freedom successively, suggesting a cascade of transitions whose underlying stability principle remains unclear. Here we formulate dimension-wise posterior collapse in variational autoencoder (VAE) as a fluctuation theory around partially collapsed states. Interpreting the negative evidence lower bound as an effective free energy, its quadratic expansion defines a Gaussian theory whose Hessian acts as a mass matrix for latent fluctuations. We show that the collapsed directions form an invariant fluctuation sector and derive its exact mass spectrum in terms of a conditional residual operator. A local reactivation direction lowers the free energy when the decoder variance falls below the residual spectral upper edge, with equality marking marginality. The criterion recovers principal component thresholds in the linear Gaussian VAE limit. Viewed in reverse along continuously connected branches, the reactivation boundary provides a local criterion for successive collapse. Numerical continuation experiments show successive loss of latent dimensions near these spectral marginalities. These results support a spectral cascade interpretation governed by residual information left unexplained by the surviving representation.
Comments19 pages, 3 figures
