YoChameleon：个性化视觉与语言生成
YoChameleon: Personalized Vision and Language Generation
AI总结：
本文提出 Yo'Chameleon，首次研究大型多模态模型的个性化，通过软提示微调机制在少样本下实现特定概念的问答与图像生成。
摘要：
大型多模态模型（如 GPT-4、Gemini、Chameleon）已发展成为拥有数百万用户的强大工具。然而，它们仍是通用模型，缺乏特定用户概念的个性化知识。先前的研究探索了文本生成的个性化，但尚不清楚如何将这些方法适配到图像生成等新模态。本文介绍了 Yo'Chameleon，这是首次对大型多模态模型个性化进行的研究。给定特定概念的 3-5 张图像，Yo'Chameleon 利用软提示微调嵌入特定主体的信息，以（i）回答关于该主体的问题，并（ii）重建像素级细节以在新语境中生成该主体的图像。Yo'Chameleon 的训练采用了（i）一种自提示优化机制来平衡多种模态间的性能，以及（ii）一种“软正样本”图像生成方法，以在少样本设置下提升图像质量。
英文摘要：
Large Multimodal Models (e.g., GPT-4, Gemini, Chameleon) have evolved into powerful tools with millions of users. However, they remain generic models and lack personalized knowledge of specific user concepts. Previous work has explored personalization for text generation, yet it remains unclear how these methods can be adapted to new modalities, such as image generation. In this paper, we introduce Yo'Chameleon, the first attempt to study personalization for large multimodal models. Given 3-5 images of a particular concept, Yo'Chameleon leverages soft-prompt tuning to embed subject-specific information to (i) answer questions about the subject and (ii) recreate pixel-level details to produce images of the subject in new contexts. Yo'Chameleon is trained with (i) a self-prompting optimization mechanism to balance performance across multiple modalities, and (ii) a ``soft-positive" image generation approach to enhance image quality in a few-shot setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
