arXiv 2102.023712021-02-07cs.CVcs.AI

MeInGame：从单个肖像创建游戏角色脸部

MeInGame: Create a Game Character Face from a Single Portrait

Netease Fuxi AI Lab（网易伏羲人工智能实验室）；University of Michigan（密歇根大学）

Jiangke Lin, Yi Yuan, Zhengxia Zou

备注：: Accepted to AAAI 2021. Code is now available at https://github.com/FuxiCV/MeInGame

摘要：

近年来，基于深度学习的三维人脸重建方法被提出，但在游戏中的应用还不多见。当前的游戏角色定制系统要么需要玩家手动调整相当多的面部属性以获得期望的面部，要么具有有限的面部形状和纹理自由度。在本文中，我们提出了一个自动的字符面部创建方法，预测面部形状和纹理从一个单一的肖像，它可以集成到大多数现有的3D游戏。虽然基于3D可变形人脸模型（3DMM）的方法可以从单个图像中恢复准确的3D人脸，但3DMM网格的拓扑结构与大多数游戏中使用的网格不同。为了获得逼真的纹理，现有的方法需要大量的人脸纹理数据进行训练，而建立这样的数据集是费时费力的。此外，在实验室条件下收集的数据集可能无法很好地推广到野外情况。为了解决这些问题，我们提出了1）一种低成本的面部纹理获取方法，2）一种可以将3DMM网格的形状转换为游戏的形状转换算法，以及3）一种用于训练3D游戏面部重建网络的新管道。该方法不仅可以生成与输入肖像相似的逼真的游戏角色，而且可以消除光照和遮挡的影响。实验表明，我们的方法优于国家的最先进的方法在游戏中使用。

英文摘要：

Many deep learning based 3D face reconstruction methods have been proposed recently, however, few of them have applications in games. Current game character customization systems either require players to manually adjust considerable face attributes to obtain the desired face, or have limited freedom of facial shape and texture. In this paper, we propose an automatic character face creation method that predicts both facial shape and texture from a single portrait, and it can be integrated into most existing 3D games. Although 3D Morphable Face Model (3DMM) based methods can restore accurate 3D faces from single images, the topology of 3DMM mesh is different from the meshes used in most games. To acquire fidelity texture, existing methods require a large amount of face texture data for training, while building such datasets is time-consuming and laborious. Besides, such a dataset collected under laboratory conditions may not generalized well to in-the-wild situations. To tackle these problems, we propose 1) a low-cost facial texture acquisition method, 2) a shape transfer algorithm that can transform the shape of a 3DMM mesh to games, and 3) a new pipeline for training 3D game face reconstruction networks. The proposed method not only can produce detailed and vivid game characters similar to the input portrait, but can also eliminate the influence of lighting and occlusions. Experiments show that our method outperforms state-of-the-art methods used in games.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文