用于零样本个性化的文本条件情境化虚拟形象
Text-Conditional Contextualized Avatars For Zero-Shot Personalization
AI总结：
本文提出一种零样本文本条件虚拟形象个性化生成流程，无需训练即可扩展至百万用户，并设计文本到3D姿态扩散模型，利用大规模图像数据学习人体姿态，显著超越现有文本到运动模型。
摘要：
近期的大规模文本到图像生成模型在合成图像的质量、真实感和多样性方面取得了显著进步，使用户能够通过语言控制生成内容。然而，这些生成模型在个性化方面仍具有挑战性且探索不足。在本工作中，我们提出了一条流程，使图像生成能够以令人愉悦的方式通过捕捉用户身份的虚拟形象实现个性化。我们的流程是零样本的，对虚拟形象的纹理和风格不敏感，并且完全不需要对虚拟形象进行训练——它可扩展到数百万用户，让用户能够用自己的虚拟形象生成场景。为了以忠实于给定文本提示的姿态渲染虚拟形象，我们提出了一种新颖的文本到3D姿态扩散模型，该模型在一个精心构建的大规模自然场景人体姿态数据集上训练，显著提升了当前最先进的文本到运动模型的性能。我们首次展示了如何利用大规模图像数据集来学习人体3D姿态参数，并克服运动捕捉数据集的局限性。
英文摘要：
Recent large-scale text-to-image generation models have made significant improvements in the quality, realism, and diversity of the synthesized images and enable users to control the created content through language. However, the personalization aspect of these generative models is still challenging and under-explored. In this work, we propose a pipeline that enables personalization of image generation with avatars capturing a user's identity in a delightful way. Our pipeline is zero-shot, avatar texture and style agnostic, and does not require training on the avatar at all - it is scalable to millions of users who can generate a scene with their avatar. To render the avatar in a pose faithful to the given text prompt, we propose a novel text-to-3D pose diffusion model trained on a curated large-scale dataset of in-the-wild human poses improving the performance of the SOTA text-to-motion models significantly. We show, for the first time, how to leverage large-scale image datasets to learn human 3D pose parameters and overcome the limitations of motion capture datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
