AvatarStudio：从文本创建高保真且可动画化的3D化身
AvatarStudio: High-fidelity and Animatable 3D Avatar Creation from Text
AI总结：
本文提出AvatarStudio，一种从粗到细的文本驱动3D化身生成模型，通过结合NeRF、SMPL关节网格和DensePose条件扩散模型，实现高保真、可动画化且姿态可控的化身创建。
摘要：
我们研究仅从文本描述创建高保真且可动画化的3D化身的问题。现有的文本到化身方法要么局限于无法动画化的静态化身，要么难以生成具有良好质量和精确姿态控制的可动画化身。为了解决这些局限性，我们提出了AvatarStudio，这是一种从粗到细的生成模型，可为可动画化的人体化身生成显式的带纹理3D网格。具体而言，AvatarStudio首先采用基于低分辨率NeRF的表示进行粗略生成，随后将SMPL引导的关节运动融入显式网格表示中，以支持化身动画和高分辨率渲染。为确保生成化身具有视图一致性和姿态可控性，我们引入了一个以DensePose为条件的2D扩散模型，用于Score Distillation Sampling监督。通过有效利用关节网格表示与DensePose条件扩散模型之间的协同作用，AvatarStudio能够从文本创建高质量且可直接用于动画的化身，显著优于以往方法。此外，它适用于许多应用，例如多模态化身动画和风格引导的化身创建。更多结果请参见我们的项目页面：http://jeff95.me/projects/avatarstudio.html
英文摘要：
We study the problem of creating high-fidelity and animatable 3D avatars from only textual descriptions. Existing text-to-avatar methods are either limited to static avatars which cannot be animated or struggle to generate animatable avatars with promising quality and precise pose control. To address these limitations, we propose AvatarStudio, a coarse-to-fine generative model that generates explicit textured 3D meshes for animatable human avatars. Specifically, AvatarStudio begins with a low-resolution NeRF-based representation for coarse generation, followed by incorporating SMPL-guided articulation into the explicit mesh representation to support avatar animation and high resolution rendering. To ensure view consistency and pose controllability of the resulting avatars, we introduce a 2D diffusion model conditioned on DensePose for Score Distillation Sampling supervision. By effectively leveraging the synergy between the articulated mesh representation and the DensePose-conditional diffusion model, AvatarStudio can create high-quality avatars from text that are ready for animation, significantly outperforming previous methods. Moreover, it is competent for many applications, e.g., multimodal avatar animations and style-guided avatar creation. For more results, please refer to our project page: http://jeff95.me/projects/avatarstudio.html
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
