HeadSculpt：用文本制作3D头部头像
HeadSculpt: Crafting 3D Head Avatars with Text
AI总结：
针对现有文本生成3D头部头像缺乏3D感知和细粒度编辑能力的问题，提出从粗到细的HeadSculpt流程，结合地标控制、文本嵌入和身份感知分数蒸馏，实现高保真生成与编辑。
摘要：
近年来，文本引导的3D生成方法在生成高质量纹理和几何形状方面取得了显著进展，这得益于大型视觉-语言模型和图像扩散模型的普及。然而，现有方法在创建高保真3D头部头像方面仍面临两个挑战：（1）它们主要依赖预训练的文本到图像扩散模型，但缺乏必要的3D感知和头部先验知识，这导致生成的头像容易出现不一致和几何变形。（2）它们在细粒度编辑方面表现不足，这主要源于预训练2D图像扩散模型的固有局限性，在涉及3D头部头像时这些局限性变得更加明显。在本工作中，我们通过引入一个名为HeadSculpt的多功能从粗到细的流程来解决这些挑战，该流程用于从文本提示制作（即生成和编辑）3D头部头像。具体来说，我们首先通过利用基于地标的控制和表示头部背面外观的学习文本嵌入来赋予扩散模型3D感知能力，从而实现3D一致的头部头像生成。我们进一步提出了一种新颖的身份感知编辑分数蒸馏策略，结合高分辨率可微渲染技术来优化纹理网格。这能够在遵循编辑指令的同时保持身份特征。通过全面实验和与现有方法的比较，我们展示了HeadSculpt在保真度和编辑能力方面的优越性。
英文摘要：
Recently, text-guided 3D generative methods have made remarkable advancements in producing high-quality textures and geometry, capitalizing on the proliferation of large vision-language and image diffusion models. However, existing methods still struggle to create high-fidelity 3D head avatars in two aspects: (1) They rely mostly on a pre-trained text-to-image diffusion model whilst missing the necessary 3D awareness and head priors. This makes them prone to inconsistency and geometric distortions in the generated avatars. (2) They fall short in fine-grained editing. This is primarily due to the inherited limitations from the pre-trained 2D image diffusion models, which become more pronounced when it comes to 3D head avatars. In this work, we address these challenges by introducing a versatile coarse-to-fine pipeline dubbed HeadSculpt for crafting (i.e., generating and editing) 3D head avatars from textual prompts. Specifically, we first equip the diffusion model with 3D awareness by leveraging landmark-based control and a learned textual embedding representing the back view appearance of heads, enabling 3D-consistent head avatar generations. We further propose a novel identity-aware editing score distillation strategy to optimize a textured mesh with a high-resolution differentiable rendering technique. This enables identity preservation while following the editing instruction. We showcase HeadSculpt's superior fidelity and editing capabilities through comprehensive experiments and comparisons with existing methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
