用户友好的多模态提示定制生成
User-Friendly Customized Generation with Multi-Modal Prompts
AI总结：
本文提出一种基于多模态提示的定制生成方法，仅需单张图像和文本即可实现复杂对象的用户友好定制，优于现有微调方法。
摘要：
文本到图像生成模型已经取得了显著的进步，满足了人们对个性化图像创作日益增长的兴趣。当前的定制技术通常需要用户为每个定制对象提供多张图像（通常为3-5张），以及对这些对象进行分类和描述场景的文本提示。本文质疑这一过程能否变得更加用户友好，以及定制能否更加精细。我们提出了一种方法，用户只需为每个定制主题提供图像和文本，并且每个视觉概念仅需一张图像。我们引入了“多模态提示”的概念，这是一种针对每个定制概念定制的文本和图像的新颖集成，简化了用户交互，并促进了对对象和场景的精确定制。我们提出的定制文本到图像生成范式在用户友好性和使用用户友好输入定制复杂对象的能力方面超越了现有的基于微调的方法。我们的代码可在 https://github.com/zhongzero/Multi-Modal-Prompt 获取。
英文摘要：
Text-to-image generation models have seen considerable advancement, catering to the increasing interest in personalized image creation. Current customization techniques often necessitate users to provide multiple images (typically 3-5) for each customized object, along with the classification of these objects and descriptive textual prompts for scenes. This paper questions whether the process can be made more user-friendly and the customization more intricate. We propose a method where users need only provide images along with text for each customization topic, and necessitates only a single image per visual concept. We introduce the concept of a ``multi-modal prompt'', a novel integration of text and images tailored to each customization concept, which simplifies user interaction and facilitates precise customization of both objects and scenes. Our proposed paradigm for customized text-to-image generation surpasses existing finetune-based methods in user-friendliness and the ability to customize complex objects with user-friendly inputs. Our code is available at $\href{https://github.com/zhongzero/Multi-Modal-Prompt}{https://github.com/zhongzero/Multi-Modal-Prompt}$.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
