arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.09326cs.GRcs.AI
收藏

HeadEvolver:通过表现力强且保持属性的网格变形实现文本到头部虚拟形象

HeadEvolver: Text to Head Avatars via Expressive and Attribute-Preserving Mesh Deformation

The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Tencent AI Lab(腾讯人工智能实验室)

Duotun Wang, Hengyu Meng, Zeyu Cai, Zhijing Shao, Qianxi Liu, Lin Wang, Mingming Fan, Xiaohang Zhan, Zeyu Wang

AI总结:

本文提出HeadEvolver框架,通过局部可学习的网格变形和2D扩散先验,从文本生成高质量且保持属性的头部虚拟形象,支持无缝编辑和动画操作。

摘要:

当前的文本到虚拟形象方法通常依赖隐式表示(如NeRF、SDF和DMTet),导致3D内容难以在图形软件中编辑和动画化。本文提出一种新颖的框架,通过文本引导生成风格化头部虚拟形象,利用局部可学习的网格变形和2D扩散先验,实现保持属性操作的高质量数字资产。给定模板网格,该方法用逐面雅可比矩阵表示网格变形,并使用可学习的向量场自适应调节局部变形。该向量场在保持顶点旋转的同时实现各向异性缩放,能更好地表达身份和几何细节。我们采用基于关键点和轮廓的正则化项,在不依赖任何特定形状先验的情况下平衡多视图下生成虚拟形象的表现力和合理性。该框架能生成可通过文本进一步编辑的真实形状和纹理,同时支持使用模板网格中保留的属性(如3DMM参数、混合形状和UV坐标)进行无缝编辑。大量实验表明,该框架能生成多样且表现力强的头部虚拟形象,其高质量网格可被艺术家在图形软件中轻松操作,促进保留属性的高效资产创建和动画等下游应用。

英文摘要:

Current text-to-avatar methods often rely on implicit representations (e.g., NeRF, SDF, and DMTet), leading to 3D content that artists cannot easily edit and animate in graphics software. This paper introduces a novel framework for generating stylized head avatars from text guidance, which leverages locally learnable mesh deformation and 2D diffusion priors to achieve high-quality digital assets for attribute-preserving manipulation. Given a template mesh, our method represents mesh deformation with per-face Jacobians and adaptively modulates local deformation using a learnable vector field. This vector field enables anisotropic scaling while preserving the rotation of vertices, which can better express identity and geometric details. We employ landmark- and contour-based regularization terms to balance the expressiveness and plausibility of generated avatars from multiple views without relying on any specific shape prior. Our framework can generate realistic shapes and textures that can be further edited via text, while supporting seamless editing using the preserved attributes from the template mesh, such as 3DMM parameters, blendshapes, and UV coordinates. Extensive experiments demonstrate that our framework can generate diverse and expressive head avatars with high-quality meshes that artists can easily manipulate in graphics software, facilitating downstream applications such as efficient asset creation and animation with preserved attributes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑