MFTF：无需掩码免训练的物体级布局控制扩散模型
MFTF: Mask-free Training-free Object Level Layout Control Diffusion Model
AI总结：
针对扩散模型仅靠文本难以精确控制物体位置且现有方法依赖额外掩码或重训练的局限，提出无需掩码免训练的物体级布局控制扩散模型（MFTF），通过源与目标扩散模型并行去噪及交叉注意力掩码引导自注意力查询调整，实现精确的单/多物体位置与语义编辑。
摘要：
文本到图像生成模型彻底改变了内容创作，但基于扩散的视觉语言模型在仅使用文本引导来精确控制生成图像中物体的形状、外观和位置放置方面仍面临挑战。现有的全局图像编辑模型依赖额外的掩码或图像作为引导来实现布局控制，通常需要对模型进行重新训练。虽然局部物体编辑模型允许修改物体形状，但它们缺乏控制物体位置的能力。为了解决这些限制，我们提出了无需掩码免训练的物体级布局控制扩散模型（MFTF），它无需额外掩码或图像即可对物体位置进行精确控制。MFTF模型支持单物体和多物体的位置调整，如平移和旋转，同时实现布局控制和物体语义编辑。MFTF模型对源扩散模型和目标扩散模型采用并行去噪过程。在此过程中，从源扩散模型的交叉注意力层动态生成注意力掩码，并将其应用于自注意力层的查询以隔离物体。在源扩散模型中生成的这些查询随后根据布局控制参数进行调整，并重新注入到目标扩散模型的自注意力层中。这种方法确保了对物体准确且精确的位置控制。项目源代码可在 https://github.com/syang-genai/MFTF 获取。
英文摘要：
Text-to-image generation models have revolutionized content creation, but diffusion-based vision-language models still face challenges in precisely controlling the shape, appearance, and positional placement of objects in generated images using text guidance alone. Existing global image editing models rely on additional masks or images as guidance to achieve layout control, often requiring retraining of the model. While local object-editing models allow modifications to object shapes, they lack the capability to control object positions. To address these limitations, we propose the Mask-free Training-free Object-Level Layout Control Diffusion Model (MFTF), which provides precise control over object positions without requiring additional masks or images. The MFTF model supports both single-object and multi-object positional adjustments, such as translation and rotation, while enabling simultaneous layout control and object semantic editing. The MFTF model employs a parallel denoising process for both the source and target diffusion models. During this process, attention masks are dynamically generated from the cross-attention layers of the source diffusion model and applied to queries from the self-attention layers to isolate objects. These queries, generated in the source diffusion model, are then adjusted according to the layout control parameters and re-injected into the self-attention layers of the target diffusion model. This approach ensures accurate and precise positional control of objects. Project source code available at https://github.com/syang-genai/MFTF.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
