从文本描述生成解耦的穿衣人体化身
Disentangled Clothed Avatar Generation from Text Descriptions
AI总结：
提出SO-SMPL解耦3D化身表示及基于SDS的蒸馏框架，分别生成人体与衣服，提升动画、试穿与编辑质量。
摘要：
在本文中，我们引入了一种新颖的文本到化身生成方法，该方法分别生成人体和衣服，并允许在生成的化身上进行高质量的动画。尽管近期文本到化身生成的进展已经能从文本提示中生成多样化的人类化身，但这些方法通常将所有元素——衣服、头发和身体——结合成单一的3D表示。这种纠缠的方法给编辑或动画等下游任务带来了挑战。为了克服这些限制，我们提出了一种名为Sequentially Offset-SMPL (SO-SMPL) 的新型解耦3D化身表示，基于SMPL模型构建。SO-SMPL使用两个独立的网格来表示人体和衣服，但通过偏移量将它们关联起来，以确保身体和衣服之间的物理对齐。然后，我们设计了一个基于Score Distillation Sampling (SDS)的蒸馏框架，用于从文本提示中生成所提出的SO-SMPL表示。我们的方法不仅实现了更高的纹理和几何质量以及与文本提示更好的语义对齐，而且还显著提升了角色动画、虚拟试穿和化身编辑的视觉质量。项目页面：https://shanemankiw.github.io/SO-SMPL/。
英文摘要：
In this paper, we introduce a novel text-to-avatar generation method that separately generates the human body and the clothes and allows high-quality animation on the generated avatar. While recent advancements in text-to-avatar generation have yielded diverse human avatars from text prompts, these methods typically combine all elements-clothes, hair, and body-into a single 3D representation. Such an entangled approach poses challenges for downstream tasks like editing or animation. To overcome these limitations, we propose a novel disentangled 3D avatar representation named Sequentially Offset-SMPL (SO-SMPL), building upon the SMPL model. SO-SMPL represents the human body and clothes with two separate meshes but associates them with offsets to ensure the physical alignment between the body and the clothes. Then, we design a Score Distillation Sampling (SDS)-based distillation framework to generate the proposed SO-SMPL representation from text prompts. Our approach not only achieves higher texture and geometry quality and better semantic alignment with text prompts, but also significantly improves the visual quality of character animation, virtual try-on, and avatar editing. Project page: https://shanemankiw.github.io/SO-SMPL/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
