Ranni：驯服文本到图像扩散模型以实现准确的指令遵循
Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Following
AI总结：
本文提出 Ranni，利用大语言模型构建语义面板并作为中间控制信号注入扩散去噪网络，以提升 T2I 模型对复杂提示的遵循能力，并支持面板调整与聊天式编辑。
摘要：
现有的文本到图像（T2I）扩散模型通常难以理解复杂提示，尤其是涉及数量、对象—属性绑定以及多主体描述的提示。在本工作中，我们引入语义面板作为将文本解码为图像过程中的中间件，以支持生成器更好地遵循指令。该面板通过在大语言模型的辅助下整理从输入文本解析出的视觉概念而获得，随后作为详细控制信号注入去噪网络，以补充文本条件。为促进文本到面板的学习，我们提出了一种精心设计的语义格式化协议，并配有全自动数据准备流程。得益于这种设计，我们称为 Ranni 的方法成功增强了预训练 T2I 生成器的文本可控性。更重要的是，生成式中间件的引入带来了更便捷的交互形式（即直接调整面板中的元素或使用语言指令），并进一步允许用户精细定制其生成结果；基于此，我们开发了一个实用系统，并展示了其在连续生成和基于聊天的编辑方面的潜力。我们的项目页面位于 https://ranni-t2i.github.io/Ranni。
英文摘要：
Existing text-to-image (T2I) diffusion models usually struggle in interpreting complex prompts, especially those with quantity, object-attribute binding, and multi-subject descriptions. In this work, we introduce a semantic panel as the middleware in decoding texts to images, supporting the generator to better follow instructions. The panel is obtained through arranging the visual concepts parsed from the input text by the aid of large language models, and then injected into the denoising network as a detailed control signal to complement the text condition. To facilitate text-to-panel learning, we come up with a carefully designed semantic formatting protocol, accompanied by a fully-automatic data preparation pipeline. Thanks to such a design, our approach, which we call Ranni, manages to enhance a pre-trained T2I generator regarding its textual controllability. More importantly, the introduction of the generative middleware brings a more convenient form of interaction (i.e., directly adjusting the elements in the panel or using language instructions) and further allows users to finely customize their generation, based on which we develop a practical system and showcase its potential in continuous generation and chatting-based editing. Our project page is at https://ranni-t2i.github.io/Ranni.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
