局部去噪的崩溃作为语义特化
Breakdown of Local Denoising as Semantic Speciation
浏览论文内容
中文总结 AI 辅助
本文通过“共同原因”假设证明生成模型中非局域窗口必位于特化窗口内，并在高斯混合中验证相变，从而用语义信息统一解释两种时间窗口的并发性。
中文摘要 AI 辅助
生成模型的动力学展现出两个明显不同的时间窗口：一个特化窗口，在此窗口中样本承诺于一个语义类别；以及一个非局域窗口，在此窗口中局部上下文窗口变得不足以进行生成。受它们在多种前沿模型中近乎同时出现的证据的启发，我们通过语义信息的空间分布来研究它们之间的关系。在“共同原因”假设下，我们证明了非局域窗口必须位于特化窗口之内。该假设假定语义标签解释了远距离标记之间相关性的一部分，这一条件对于许多真实数据集是自然的。我们进一步给出了当系统规模增长时两个窗口收缩到单一极限时间的条件，定义了一个“相变”，并在高斯混合模型中解析验证了这一行为。综合来看，这些结果确定了语义信息解释特化与非局域同时出现的条件，连接了生成建模中语义结构涌现的两个互补视角。
英文摘要
The dynamics of generative models exhibit two apparently distinct temporal windows: a speciation window, in which a sample commits to a semantic class, and a nonlocality window, in which local context windows become insufficient for generation. Motivated by evidence of their near-concurrence in a variety of frontier models, we investigate their relationship through the spatial distribution of semantic information. Under a "common cause" hypothesis, we prove that the nonlocality window must lie in the speciation window. This hypothesis postulates that semantic labels explain a fraction of the correlations between distant tokens, a condition that is natural for many real datasets. We further give conditions under which both windows shrink to a single limiting time as system size grows, defining a "phase transition", and verify this behavior analytically in Gaussian mixtures. Together, these results identify conditions under which semantic information explains the concurrence of speciation and nonlocality, connecting two complementary perspectives on the emergence of semantic structure in generative modeling.
发表机构
- University of Colorado Boulder（科罗拉多大学博尔德分校）
- Princeton University（普林斯顿大学）
- QuEra Computing Inc.（QuEra计算公司）
机构由 AI 辅助整理，请以论文原文为准。
