用于组合式图像生成的对象级视觉提示
Object-level Visual Prompts for Compositional Image Generation
AI总结：
本文提出在文本到图像扩散模型中组合对象级视觉提示，通过 KV 混合交叉注意力机制和对象级组合引导，在保持对象身份的同时生成多样化场景组合。
摘要：
我们提出了一种在文本到图像扩散模型中组合对象级视觉提示的方法。我们的方法针对的任务是：在多样化的场景和风格中生成语义一致的组合，类似于文本提示所提供的通用性和表现力。该任务的一个关键挑战在于，既要保持输入视觉提示中所描绘对象的身份，又要在不同图像之间生成多样化的组合。为了解决这一挑战，我们引入了一种新的 KV 混合交叉注意力机制，其中键（keys）和值（values）从不同的视觉表示中学习。键来自一个具有小瓶颈的编码器，用于布局控制；而值来自一个具有较大瓶颈的编码器，用于捕获细粒度的外观细节。通过混合来自这些互补来源的键和值，我们的模型在保持视觉提示对象身份的同时，支持在对象排列、姿态和构图上的灵活变化。在推理过程中，我们进一步提出对象级组合引导，以改善该方法的身份保持和布局正确性。结果表明，我们的技术能够生成多样化的场景组合，同时保留每个视觉提示的独特特征，从而扩展了文本到图像生成的创作潜力。
英文摘要：
We introduce a method for composing object-level visual prompts within a text-to-image diffusion model. Our approach addresses the task of generating semantically coherent compositions across diverse scenes and styles, similar to the versatility and expressiveness offered by text prompts. A key challenge in this task is to preserve the identity of the objects depicted in the input visual prompts, while also generating diverse compositions across different images. To address this challenge, we introduce a new KV-mixed cross-attention mechanism, in which keys and values are learned from distinct visual representations. The keys are derived from an encoder with a small bottleneck for layout control, whereas the values come from a larger bottleneck encoder that captures fine-grained appearance details. By mixing keys and values from these complementary sources, our model preserves the identity of the visual prompts while supporting flexible variations in object arrangement, pose, and composition. During inference, we further propose object-level compositional guidance to improve the method's identity preservation and layout correctness. Results show that our technique produces diverse scene compositions that preserve the unique characteristics of each visual prompt, expanding the creative potential of text-to-image generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
