在哪里以及如何扰动：关于扩散和流模型中扰动引导的设计
Where and How to Perturb: On the Design of Perturbation Guidance in Diffusion and Flow Models
AI总结：
针对扩散模型中注意力扰动缺乏原则性设计的问题，提出HeadHunter框架选择特定注意力头以细粒度控制生成质量，并引入SoftPAG线性插值调节扰动强度，在Stable Diffusion 3和FLUX.1上实现了卓越的图像质量增强与风格引导。
摘要：
扩散模型中最近的引导方法通过扰动模型来构建隐式弱模型，并引导生成远离它，从而控制反向采样。在这些方法中，注意力扰动在无分类器引导不适用的无条件场景中展现出了强大的经验性能。然而，现有的注意力扰动方法缺乏确定应在何处应用扰动的原则性方法，特别是在质量相关计算分布在各层的Diffusion Transformer (DiT)架构中。在本文中，我们研究了注意力扰动的粒度，范围从层级别一直到单个注意力头，并发现特定的头控制着不同的视觉概念，如结构、风格和纹理质量。基于这一见解，我们提出了“HeadHunter”，这是一个用于迭代选择与以用户为中心的目标对齐的注意力头的系统框架，能够对生成质量和视觉属性进行细粒度控制。此外，我们引入了SoftPAG，它将每个选定头的注意力图向单位矩阵进行线性插值，提供了一个连续的旋钮来调节扰动强度并抑制伪影。我们的方法不仅缓解了现有层级扰动过度平滑的问题，还通过组合头选择实现了对特定视觉风格的定向操纵。我们在基于现代大规模DiT的文本到图像模型上验证了我们的方法，包括Stable Diffusion 3和FLUX.1，证明了其在一般质量增强和风格特定引导方面的卓越性能。我们的工作提供了扩散模型中注意力扰动的首个头级别分析，揭示了注意力层内可解释的特化性，并实现了有效扰动策略的实用设计。
英文摘要：
Recent guidance methods in diffusion models steer reverse sampling by perturbing the model to construct an implicit weak model and guide generation away from it. Among these approaches, attention perturbation has demonstrated strong empirical performance in unconditional scenarios where classifier-free guidance is not applicable. However, existing attention perturbation methods lack principled approaches for determining where perturbations should be applied, particularly in Diffusion Transformer (DiT) architectures where quality-relevant computations are distributed across layers. In this paper, we investigate the granularity of attention perturbations, ranging from the layer level down to individual attention heads, and discover that specific heads govern distinct visual concepts such as structure, style, and texture quality. Building on this insight, we propose "HeadHunter", a systematic framework for iteratively selecting attention heads that align with user-centric objectives, enabling fine-grained control over generation quality and visual attributes. In addition, we introduce SoftPAG, which linearly interpolates each selected head's attention map toward an identity matrix, providing a continuous knob to tune perturbation strength and suppress artifacts. Our approach not only mitigates the oversmoothing issues of existing layer-level perturbation but also enables targeted manipulation of specific visual styles through compositional head selection. We validate our method on modern large-scale DiT-based text-to-image models including Stable Diffusion 3 and FLUX.1, demonstrating superior performance in both general quality enhancement and style-specific guidance. Our work provides the first head-level analysis of attention perturbation in diffusion models, uncovering interpretable specialization within attention layers and enabling practical design of effective perturbation strategies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
