看哪里不等于如何修复：扩散组合中的预去噪诊断与模态依赖控制
Where to Look Is Not How to Fix: Pre-Denoising Diagnostics and Modality-Dependent Control in Diffusion Composition
浏览论文内容
中文总结 AI 辅助
本研究通过受控协议揭示扩散模型组合缺陷在去噪前可诊断，但诊断坐标与改善生成的干预坐标存在分离，并提出组合压力指数及模态依赖控制策略。
中文摘要 AI 辅助
理解文本到图像扩散中的组合失败，需要同时识别压力在何处可检测以及干预如何改变输出。我们通过一个受控的锚点-压力协议来研究这些问题，该协议联合评估文本编码器诊断和去噪器干预。我们引入了一个纯文本的组合压力指数（CSI），该指数在SD1.5、SDXL和SD3文本路径上区分常见与罕见组合，并提供了一个上游诊断坐标。一项匹配的六提示定位研究将干预位置与不同结果联系起来：残差最小化嵌入适配器改善了表示拟合，而下游交叉注意力干预使颜色命中率（CHR）提高了0.0272。在SD1.5和SDXL去噪器块中，最大的正符号诊断可达性均值出现在深层编码器处，而选择性增强在解码器块处具有最大的正平均CHR响应。选择性减法和广泛消融揭示了进一步的模态和架构依赖响应，包括当SDXL解码器交叉注意力被广泛消融时CHR显著下降。我们在受控的属性-对象组合设置下发现了一种诊断-控制分离：组合缺陷在去噪前是可诊断的，但暴露风险的表示坐标不一定是改善生成的坐标或模态。
英文摘要
Understanding compositional failures in text-to-image diffusion requires identifying both where stress is detectable and how intervention changes the output. We study these questions through a controlled anchor--stress protocol that jointly evaluates text-encoder diagnostics and denoiser interventions. We introduce a text-only Compositional Stress Index (CSI), which separates common from rare compositions across SD1.5, SDXL, and the SD3 text path and provides an upstream diagnostic coordinate. A matched six-prompt localization study links intervention location to distinct outcomes: residual-minimizing embedding adapters improve representation fit, while downstream cross-attention intervention increases color hit rate (CHR) by 0.0272. Across SD1.5 and SDXL denoiser blocks, the largest positive signed diagnostic-accessibility mean occurs at the deep encoder, whereas selective boost has its largest positive mean CHR response at decoder blocks. Selective subtraction and broad ablation reveal further modality- and architecture-dependent responses, including a substantial CHR decrease when SDXL decoder cross-attention is broadly ablated. We find a diagnosis-control dissociation under our controlled attribute-object composition setting: compositional defects are diagnosable before denoising, but the representation coordinate that exposes risk is not necessarily the coordinate or modality that improves generation.
发表机构
- Tulane University（杜兰大学）
机构由 AI 辅助整理，请以论文原文为准。
