从扩散模型中引入多样性以实现语义引导的面部资产生成
Bringing Diversity from Diffusion Models to Semantic-Guided Face Asset Generation
AI总结：
本文提出了一种语义可控的生成网络，通过扩散模型生成高质量3D面部数据库，并开发了高效的GAN基生成器，用于创建和编辑高质量面部资产。
摘要：
数字建模和重建人类面部服务于各种应用。然而，其可用性常常受到数据捕获设备的要求、人工劳动和合适演员的限制。这种状况限制了生成模型的多样性、表达性和控制能力。本工作旨在证明一个语义可控的生成网络可以为数字面部建模过程提供增强的控制。为了在受控环境下有限的人脸扫描数据之外增强多样性，我们引入了一种新的数据生成管道，利用预训练的扩散模型生成高质量的3D面部数据库。我们提出的归一化模块将扩散模型合成的数据转换为高质量的扫描数据。利用我们获得的44,000个面部模型，我们进一步开发了高效的GAN基生成器。该生成器接受语义属性作为输入，生成几何和光谱。它还允许在潜在空间中对属性进行连续后编辑。我们的资产精炼组件随后创建基于物理的面部资产。我们介绍了一个全面的系统，用于创建和编辑高质量的面部资产。我们提出的模型已进行了广泛的实验、比较和评估。我们还将一切整合到一个基于网页的交互工具中。我们旨在在论文发布时使该工具公开可用。
英文摘要：
Digital modeling and reconstruction of human faces serve various applications. However, its availability is often hindered by the requirements of data capturing devices, manual labor, and suitable actors. This situation restricts the diversity, expressiveness, and control over the resulting models. This work aims to demonstrate that a semantically controllable generative network can provide enhanced control over the digital face modeling process. To enhance diversity beyond the limited human faces scanned in a controlled setting, we introduce a novel data generation pipeline that creates a high-quality 3D face database using a pre-trained diffusion model. Our proposed normalization module converts synthesized data from the diffusion model into high-quality scanned data. Using the 44,000 face models we obtained, we further developed an efficient GAN-based generator. This generator accepts semantic attributes as input, and generates geometry and albedo. It also allows continuous post-editing of attributes in the latent space. Our asset refinement component subsequently creates physically-based facial assets. We introduce a comprehensive system designed for creating and editing high-quality face assets. Our proposed model has undergone extensive experiment, comparison and evaluation. We also integrate everything into a web-based interactive tool. We aim to make this tool publicly available with the release of the paper.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
