基于深度学习的二值模板图像人体形状重建
Shape-from-Mask: A Deep Learning Based Human Body Shape Reconstruction from Binary Mask Images
摘要:
3D内容创建被称为计算机图形学最基本的任务之一。在过去的几十年里,人们开发了许多基于2D图像或曲线的3D建模算法。设计师可以从正面、侧面和顶视图对齐一些概念图像或绘制一些暗示性曲线,然后将它们用作自动或手动构建3D模型的参考。然而,据我们所知,还没有研究以类似的方式研究3D人体重建。在本文中,我们提出了一种基于深度学习的三维人体形状重建从二维正交视图。设计了一种基于CNN的回归网络,该网络具有两个分支,分别对应于正面和侧面视图,用于从2D掩模图像估计3D人体形状。我们分别训练我们的网络,以解耦的特征描述符编码的身体参数从不同的意见,并融合它们来估计一个准确的人体形状。此外,针对训练数据不足的问题,本文提出了一些针对三维人体形状的数据增强方案,以促进这一课题的进一步研究。大量的实验结果表明,使用我们的算法可以有效地实现视觉逼真和准确的重建。该方法只需要二值掩模图像,可以帮助用户快速创建自己的数字化身,也可以方便地创建用于3D游戏、虚拟现实、在线时尚购物的数字人体。
英文摘要:
3D content creation is referred to as one of the most fundamental tasks of computer graphics. And many 3D modeling algorithms from 2D images or curves have been developed over the past several decades. Designers are allowed to align some conceptual images or sketch some suggestive curves, from front, side, and top views, and then use them as references in constructing a 3D model automatically or manually. However, to the best of our knowledge, no studies have investigated on 3D human body reconstruction in a similar manner. In this paper, we propose a deep learning based reconstruction of 3D human body shape from 2D orthographic views. A novel CNN-based regression network, with two branches corresponding to frontal and lateral views respectively, is designed for estimating 3D human body shape from 2D mask images. We train our networks separately to decouple the feature descriptors which encode the body parameters from different views, and fuse them to estimate an accurate human body shape. In addition, to overcome the shortage of training data required for this purpose, we propose some significantly data augmentation schemes for 3D human body shapes, which can be used to promote further research on this topic. Extensive experimen- tal results demonstrate that visually realistic and accurate reconstructions can be achieved effectively using our algorithm. Requiring only binary mask images, our method can help users create their own digital avatars quickly, and also make it easy to create digital human body for 3D game, virtual reality, online fashion shopping.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。