在教师引导的VAE中的一种可测试的常数坍缩证书
A Testable Certificate for Constant Collapse in Teacher-Guided VAEs
AI总结：
本文研究了教师引导VAE中输入无关的常数坍缩问题，提出了一种可测量的证书，通过教师互信息界定坍缩边界，并在实验中验证了证书的有效性。
摘要：
变分自编码器中的后验坍缩通常通过其症状诊断：小的KL项、强大的解码器或弱的潜在代码使用。这些信号有用，但它们不定义坍缩边界。我们研究了一种具体的失败模式，即输入无关的常数坍缩，并表明这种情况允许精确的阈值。对于任何固定的非常数教师分布T(·|x)，最佳的常数学生是数据集平均教师分布，其对齐成本是教师互信息I_T(X;T)。因此，如果一个严格仅依赖潜在的原始见证者在对齐损失低于此值时具有安全余量，则该见证者不能在输入上是常数。这一身份将一种定性失败模式转化为可测量的模式。在CIFAR-100实验中，使用每种子教师搜索，完整训练始终位于证书边界的一侧，去除对齐会将原始见证者驱动到常数学生模式，并从坍缩检查点重新启动并启用对齐可恢复证书。Tiny-ImageNet-200固定目标运行显示，这种预防-坍缩-救援模式在三个独立搜索的教师中一致出现。标准VAE风格的基线方法，包括保留重建质量或事后可预测性的方法，仍保持在原始证书的负侧。这种保证是故意狭窄的：它证明了匹配的非常数教师相对变化通过潜在路径，而不是声称所有形式的后验坍缩都被排除。
英文摘要：
Posterior collapse in variational autoencoders is often diagnosed by its symptoms: a small KL term, a strong decoder, or weak use of the latent code. These signals are useful, but they do not define a collapse boundary. We study a concrete failure mode, input-independent constant collapse, and show that this case admits an exact threshold. For any fixed nonconstant teacher distribution \(T(\cdot\mid x)\), the best constant student is the dataset-average teacher distribution, and its alignment cost is the teacher mutual information \(I_T(X;T)\). Therefore, if a strictly latent-only raw witness achieves alignment loss below this value, with a safety margin, the witness cannot be constant in the input. This identity turns a qualitative failure mode into a measurable one. In CIFAR-100 experiments with per-seed teacher search, full training stays on the certified side of the boundary, removing alignment drives the raw witness into the constant-student regime, and restarting from a collapsed checkpoint with alignment enabled restores the certificate. Tiny-ImageNet-200 fixed-target runs show the same prevention--collapse--rescue pattern across three independently searched teachers. Standard VAE-style baselines, including methods that preserve reconstruction quality or post-hoc predictability, remain negative under the raw certificate. The guarantee is intentionally narrow: it certifies that the matched nonconstant teacher-relative variation passes through the latent pathway, rather than claiming that all forms of posterior collapse have been ruled out.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
