扩散模型能否学习图像背后隐藏的特征间规则？
Can Diffusion Models Learn Hidden Inter-Feature Rules Behind Images?
AI总结：
本文研究扩散模型学习图像特征间隐藏规则的能力，发现其在细粒度规则上存在一致失败，并通过理论分析指出DSM目标与规则一致性不兼容，最后提出分类器引导采样方法以有限改进规则学习。
摘要：
尽管扩散模型（DMs）在数据生成方面取得了显著成功，但它们仍表现出特定的失败案例，输出结果不尽如人意。我们聚焦于其中一个局限性：DMs学习图像特征之间隐藏规则的能力。具体而言，对于具有依赖特征（$\mathbf{x}$）和（$\mathbf{y}$）的图像数据（例如，太阳高度（$\mathbf{x}$）与阴影长度（$\mathbf{y}$）），我们研究DMs能否准确捕捉特征间规则（$p(\mathbf{y}|\mathbf{x})$）。在主流DMs（如Stable Diffusion 3.5）上的实证评估揭示了一致的失败，例如光照与阴影关系不一致以及物体与镜面反射不匹配。受这些发现启发，我们设计了四个具有强相关特征的合成任务，以评估DMs的规则学习能力。大量实验表明，虽然DMs能够识别粗粒度规则，但在细粒度规则上存在困难。我们的理论分析证明，通过去噪得分匹配（DSM）训练的DMs在学习隐藏规则时表现出恒定误差，因为DSM目标与规则一致性不兼容。为缓解这一问题，我们引入了一种常用技术——在采样过程中加入额外的分类器引导，这取得了（有限的）改进。我们的分析揭示，细粒度规则的细微信号对分类器而言难以捕捉，这为未来探索提供了见解。
英文摘要：
Despite the remarkable success of diffusion models (DMs) in data generation, they exhibit specific failure cases with unsatisfactory outputs. We focus on one such limitation: the ability of DMs to learn hidden rules between image features. Specifically, for image data with dependent features ($\mathbf{x}$) and ($\mathbf{y}$) (e.g., the height of the sun ($\mathbf{x}$) and the length of the shadow ($\mathbf{y}$)), we investigate whether DMs can accurately capture the inter-feature rule ($p(\mathbf{y}|\mathbf{x})$). Empirical evaluations on mainstream DMs (e.g., Stable Diffusion 3.5) reveal consistent failures, such as inconsistent lighting-shadow relationships and mismatched object-mirror reflections. Inspired by these findings, we design four synthetic tasks with strongly correlated features to assess DMs' rule-learning abilities. Extensive experiments show that while DMs can identify coarse-grained rules, they struggle with fine-grained ones. Our theoretical analysis demonstrates that DMs trained via denoising score matching (DSM) exhibit constant errors in learning hidden rules, as the DSM objective is not compatible with rule conformity. To mitigate this, we introduce a common technique - incorporating additional classifier guidance during sampling, which achieves (limited) improvements. Our analysis reveals that the subtle signals of fine-grained rules are challenging for the classifier to capture, providing insights for future exploration.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
