定制视觉:通过个性化提示重写增强文本到图像生成
Tailored Visions: Enhancing Text-to-Image Generation with Personalized Prompt Rewriting
AI总结:
本文提出基于30万条用户提示数据训练个性化提示重写模型,通过历史交互增强文本到图像生成中用户提示的表达力与一致性,并在离线和在线测试中验证其优于基线方法。
摘要:
尽管该领域已取得显著进展,创建与个体用户愿望和偏好紧密契合的个性化视觉表示仍然具有挑战性。这一过程要求用户用既能被模型理解、又能准确捕捉其视觉构想的语言来表达想法,这对许多用户而言存在困难。本文通过利用用户与系统的历史交互来增强用户提示,以应对这一挑战。我们提出了一种新方法,基于新收集的大规模文本到图像数据集(包含来自3115名用户的超过30万条提示)对用户提示进行重写。我们的重写模型增强了用户提示的表达力及其与预期视觉输出之间的一致性。实验结果证明了我们的方法相对于基线方法的优越性,这在我们新的离线评估方法和在线测试中均得到验证。我们的代码和数据集可在 https://github.com/zzjchen/Tailored-Visions 获取。
英文摘要:
Despite significant progress in the field, it is still challenging to create personalized visual representations that align closely with the desires and preferences of individual users. This process requires users to articulate their ideas in words that are both comprehensible to the models and accurately capture their vision, posing difficulties for many users. In this paper, we tackle this challenge by leveraging historical user interactions with the system to enhance user prompts. We propose a novel approach that involves rewriting user prompts based on a newly collected large-scale text-to-image dataset with over 300k prompts from 3115 users. Our rewriting model enhances the expressiveness and alignment of user prompts with their intended visual outputs. Experimental results demonstrate the superiority of our methods over baseline approaches, as evidenced in our new offline evaluation method and online tests. Our code and dataset are available at https://github.com/zzjchen/Tailored-Visions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。