arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.01406cs.CV
收藏

HumanNorm:学习法向扩散模型以实现高质量逼真三维人体生成

HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generation

Northwestern Polytechnical University(西北工业大学);Tsinghua University(清华大学)

Xin Huang, Ruizhi Shao, Qi Zhang, Hongwen Zhang, Ying Feng, Yebin Liu, Qing Wang

AI总结:

HumanNorm通过法向自适应与法向对齐扩散模型增强三维几何感知,并结合渐进几何生成策略与多步SDS损失,实现高质量逼真三维人体生成。

摘要:

近年来,采用扩散模型的文本到三维方法在三维人体生成方面取得了显著进展。然而,由于文本到图像扩散模型缺乏对三维结构的理解,这些方法面临诸多挑战,难以实现高质量的人体生成,导致几何形状平滑、外观卡通化。本文提出HumanNorm,一种面向高质量逼真三维人体生成的新方法。其核心思想是通过学习法向自适应扩散模型和法向对齐扩散模型,增强模型对三维几何的二维感知能力。法向自适应扩散模型能够根据用户提示生成高保真法向图,并支持视角相关和身体感知的文本描述。法向对齐扩散模型学习生成与法向图对齐的彩色图像,从而将物理几何细节转化为逼真外观。利用所提出的法向扩散模型,我们设计了一种渐进式几何生成策略和多步分数蒸馏采样(SDS)损失,以提升三维人体生成的性能。综合实验验证了HumanNorm能够生成具有精细几何结构和逼真外观的三维人体。在几何质量和纹理质量方面,HumanNorm均优于现有的文本到三维方法。HumanNorm的项目主页为https://humannorm.github.io/。

英文摘要:

Recent text-to-3D methods employing diffusion models have made significant advancements in 3D human generation. However, these approaches face challenges due to the limitations of text-to-image diffusion models, which lack an understanding of 3D structures. Consequently, these methods struggle to achieve high-quality human generation, resulting in smooth geometry and cartoon-like appearances. In this paper, we propose HumanNorm, a novel approach for high-quality and realistic 3D human generation. The main idea is to enhance the model's 2D perception of 3D geometry by learning a normal-adapted diffusion model and a normal-aligned diffusion model. The normal-adapted diffusion model can generate high-fidelity normal maps corresponding to user prompts with view-dependent and body-aware text. The normal-aligned diffusion model learns to generate color images aligned with the normal maps, thereby transforming physical geometry details into realistic appearance. Leveraging the proposed normal diffusion model, we devise a progressive geometry generation strategy and a multi-step Score Distillation Sampling (SDS) loss to enhance the performance of 3D human generation. Comprehensive experiments substantiate HumanNorm's ability to generate 3D humans with intricate geometry and realistic appearances. HumanNorm outperforms existing text-to-3D methods in both geometry and texture quality. The project page of HumanNorm is https://humannorm.github.io/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑