发表机构
Westlake University; Zhejiang University; Australian National University（西湖大学; 浙江大学; 澳大利亚国立大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出特征信息动力学框架，利用I-MMSE恒等式估计扩散过程中特征生成的时间，并验证了像素扩散中的谱自回归及不同表示下的信息密度差异，表明有序生成有益于训练扩散模型。
AI 中文摘要
扩散模型通过一系列去噪问题生成数据，并被广泛观察到在精细细节之前揭示粗结构。然而，这种直觉大多是经验性的和定性的。我们引入了特征信息动力学，这是一个信息论框架，用于定位特征在扩散过程中何时被生成。利用I-MMSE恒等式，我们将特征互信息变化率与最优无条件去噪损失和特征条件去噪损失之间的差距联系起来，从而为特征信息密度提供了实用的估计器。我们进一步开发了一种链式分解，将特征层级中的共享信息与增量信息分离。我们首先使用该框架定量确认像素扩散中的谱自回归，然后将分析扩展到频率之外：在类别→掩码→Canny条件链下，每个特征的信息密度在像素、SDVAE、VAVAE和RAE之间有所不同，揭示了这些表示之间的根本差异，并表明有序生成可能有利于训练扩散模型。我们的代码可在以下网址获取：https://this https URL。
英文摘要
Diffusion models generate data through a continuum of denoising problems, and are widely observed to reveal coarse structure before fine detail. Yet, this intuition is mostly empirical and qualitative. We introduce feature information dynamics, an information-theoretic framework for localizing when a feature is generated during diffusion. Using the I-MMSE identity, we connect the rate of feature mutual information change to a gap between optimal unconditional and feature-conditional denoising losses, yielding practical estimators for feature information density. We further develop a chained decomposition that separates shared from incremental information in a feature hierarchy. We use this framework first to quantitatively confirm spectral autoregression in pixel diffusion, and then to extend the analysis beyond frequency: under a class $\to$ mask $\to$ Canny conditioning chain, the per-feature information densities differ across pixel, SDVAE, VAVAE, and RAE, exposing fundamental differences between these representations and suggesting that ordered generation could be beneficial for training diffusion models. Our code is available at https://github.com/AI4Science-WestlakeU/feature-information-dynamics.
CommentsAccepted as poster at NeurIPS 2026. 28 pages, including references, appendices, and checklist
