多主体图像生成的层次化概念到外观引导
Hierarchical Concept-to-Appearance Guidance for Multi-Subject Image Generation
AI总结：
提出层次化概念到外观引导框架(CAG)，通过VAE dropout训练和对应感知掩码注意力模块，实现多主体图像生成中身份一致性和精确组合控制。
摘要：
多主体图像生成旨在合成图像，该图像在遵循文本指令的同时忠实保留多个参考主体的身份。然而，现有方法通常面临身份不一致和组合控制有限的问题，因为它们依赖扩散模型隐式地将文本提示与参考图像关联。在这项工作中，我们提出了层次化概念到外观引导（CAG），一个从高层概念到细粒度外观提供显式结构化监督的框架。在概念层面，我们引入VAE dropout训练策略，随机省略参考VAE特征，鼓励模型更多地依赖来自视觉语言模型（VLM）的稳健语义信号，从而在缺乏完整外观线索时促进一致的概念级生成。在外观层面，我们将VLM导出的对应关系集成到扩散变压器（DiT）内的对应感知掩码注意力模块中。该模块限制每个文本标记仅关注其匹配的参考区域，确保精确的属性绑定和可靠的多主体组合。大量实验表明，我们的方法在多主体图像生成上达到了最先进的性能，显著提升了提示遵循和主体一致性。
英文摘要：
Multi-subject image generation aims to synthesize images that faithfully preserve the identities of multiple reference subjects while following textual instructions. However, existing methods often suffer from identity inconsistency and limited compositional control, as they rely on diffusion models to implicitly associate text prompts with reference images. In this work, we propose Hierarchical Concept-to-Appearance Guidance (CAG), a framework that provides explicit, structured supervision from high-level concepts to fine-grained appearances. At the conceptual level, we introduce a VAE dropout training strategy that randomly omits reference VAE features, encouraging the model to rely more on robust semantic signals from a Visual Language Model (VLM) and thereby promoting consistent concept-level generation in the absence of complete appearance cues. At the appearance level, we integrate the VLM-derived correspondences into a correspondence-aware masked attention module within the Diffusion Transformer (DiT). This module restricts each text token to attend only to its matched reference regions, ensuring precise attribute binding and reliable multi-subject composition. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the multi-subject image generation, substantially improving prompt following and subject consistency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
