扩散模型即使在得分函数不敏感的情况下也能恢复准确的混合权重
Diffusion models recover accurate mixture weights despite score function insensitivity
浏览论文内容
中文总结 AI 辅助
研究基于得分的生成模型在多模态分布中恢复混合权重的问题，通过定义扩散得分敏感性指数，证明其控制目标分布参数估计准确性，还展示了噪声调度对敏感性及模式放大的影响，此框架可用于恢复目标分布的定性参数。
中文摘要 AI 辅助
基于得分的生成模型存在一个令人困惑的行为：它们似乎能覆盖目标多模态分布的所有模式，但可能无法学习到正确的相对模式幅度，即混合权重。我们通过将扩散得分匹配（DSM）损失与从生成样本估计混合权重的误差联系起来解决了这个明显的悖论。我们表明，即使目标得分对混合权重不敏感，如果中间噪声水平的得分能提供有关权重的信息，生成样本也能准确恢复权重。我们定义了扩散得分敏感性指数（DSSI），并证明它控制着从生成样本估计目标分布参数的准确性。对于任意维度的高斯混合，我们证明在温和条件下混合权重估计误差与DSM损失处于同一量级。实证上，我们展示了在典型噪声调度下基准数据分布的噪声过程中敏感性的出现，并且这些敏感性值预测了训练良好的模型恢复混合权重的程度。此外，我们表明噪声调度的选择可以降低扩散敏感性，导致模式放大。虽然我们专注于混合权重，但所提出的敏感性框架控制着目标分布任何定性参数的恢复。
英文摘要
Score-based generative models exhibit a puzzling behavior: they often appear to cover all modes of a target multimodal distribution and yet may fail to learn the correct relative mode amplitudes, which can be interpreted as mixture weights. We resolve this apparent paradox by relating the diffusion score matching (DSM) loss to the error in estimating mixture weights from generated samples. We show that, even when the target score is insensitive to mixture weights, generated samples can recover the weights accurately if the scores at intermediate noise levels are informative about the weights. Accordingly, we define the diffusion score sensitivity index (DSSI) as the variation in the DSM loss relative to changes in a parameter. We then show that the DSSI governs the accuracy with which the parameter of the target distribution can be estimated from generated samples. For Gaussian mixtures in arbitrary dimensions, we prove that the mixture weight estimation errors are on the same order as the DSM loss under mild conditions. Empirically, we show the emergence of sensitivity during the noising process of benchmark data distributions under typical noise schedules, and that these sensitivity values predict how well a well-trained model recovers mixture weights. Furthermore, we show that the choice of noise schedule can reduce diffusion sensitivity, leading to mode amplification. Although we focus on mixture weights, the proposed sensitivity framework governs the recovery of any qualitative parameter of the target distribution.
发表机构
- University of Chicago（芝加哥大学）
- Data Science Institute（数据科学研究所）
- Department of Computer Science（计算机科学系）
机构由 AI 辅助整理，请以论文原文为准。
