扩散模型中的角色偏差：通过中间分解进行诊断与缓解
Role Bias in Diffusion Models: Diagnosing and Mitigating through Intermediate Decomposition
AI总结：
本文提出 RoleBench 基准和 ReBind 框架，通过中间组合分解与微调缓解扩散模型中的角色坍缩问题，显著提升组合泛化能力。
摘要：
文本到图像（T2I）扩散模型展现出令人印象深刻的逼真图像生成能力，但在组合式图像生成方面仍存在困难。在本工作中，我们引入了 RoleBench，这是一个专注于评估基于动作关系的组合泛化能力的基准（例如，“老鼠追猫”）。我们表明，最先进的 T2I 模型和组合生成方法始终默认生成频繁的反向关系（即“猫追老鼠”），我们将这一现象称为角色坍缩。相关工作将其归因于模型的架构限制或数据中的代表性不足。我们的关键洞察揭示，虽然模型在稀有组合上失败而其反向组合很常见时，它们却能够成功生成类似的中间组合（例如，“老鼠追男孩”），这表明该限制也源于频繁对应组合的存在，而不仅仅是稀有组合的缺失。受此启发，我们假设方向性分解可以逐步缓解角色坍缩。我们通过 ReBind 来验证这一点，ReBind 是一个轻量级框架，利用精心选择的主动/被动中间组合来教授角色绑定。实验表明，通过简单的微调，中间组合可以显著减少角色坍缩，与最先进的方法相比，人类对 ReBind 的偏好超过 78%。我们的发现突显了分布不对称在组合失败中的作用，并为改善泛化提供了一条简单有效的路径。
英文摘要：
Text-to-image (T2I) diffusion models exhibit impressive photorealistic image generation capabilities, yet they struggle in compositional image generation. In this work, we introduce RoleBench, a benchmark focused on evaluating compositional generalization in action-based relations (e.g., "mouse chasing cat"). We show that state-of-the-art T2I models and compositional generation methods consistently default to frequent reversed relations (i.e., "cat chasing mouse"), a phenomenon we call role collapse. Related works attribute this to the model's architectural limitation or underrepresentation in the data. Our key insight reveals that while models fail on rare compositions when their inversions are common, they can successfully generate similar intermediate compositions (e.g., "mouse chasing boy"), suggesting that this limitation is also due to the presence of frequent counterparts rather than just the absence of rare compositions. Motivated by this, we hypothesize that directional decomposition can gradually mitigate role collapse. We test this via ReBind, a lightweight framework that teaches role bindings using carefully selected active/passive intermediate compositions. Experiments suggest that intermediate compositions through simple fine-tuning can significantly reduce role collapse, with humans preferring ReBind more than 78% compared to state-of-the-art methods. Our findings highlight the role of distributional asymmetries in compositional failures and offer a simple, effective path for improving generalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
