从图画地图上的人体图形推断隐式3D表示
Inferring Implicit 3D Representations from Human Figures on Pictorial Maps
AI总结：
本研究提出一种自动化工作流程，利用神经网络从图画地图上的单视图人体图形推断3D隐式表示，通过深度估计、SDF推断、纹理生成与补全等步骤生成3D模型，可用于数字3D地图的动画与故事讲述。
摘要：
在这项工作中，我们提出了一种自动化工作流程，将图画地图上最常出现的实体之一——人体图形——带入三维空间。我们的工作流程基于训练数据和用于从照片进行真实人体单视图3D重建的神经网络。我们首先让一个由全连接层组成的网络估计2D姿态点的深度坐标。获得的3D姿态点连同身体部位的2D掩码一起输入到一个深度隐式表面网络中，以推断3D符号距离场（SDFs）。通过组装所有身体部位，我们得出整个人体图形在不同视图下的2D深度图像和身体部位掩码，并将其输入到一个全卷积网络中以预测UV图像。这些UV图像和给定视角的纹理被插入到一个生成网络中，以对其他视图的纹理进行补全。纹理通过卡通化网络进行增强，面部细节由自编码器重新合成。最后，生成的纹理在光线行进器中被分配给推断出的身体部位。在验证了多种网络配置后，我们用12个图画人体图形测试了我们的工作流程。创建的3D模型总体上看起来很有前景，特别是考虑到基于轮廓的3D恢复和隐式SDFs的实时渲染所带来的挑战。仍需进一步改进以减少身体部位之间的间隙并为纹理添加图画细节。总体而言，构建的人体图形可用于数字3D地图中的动画和故事讲述。
英文摘要：
In this work, we present an automated workflow to bring human figures, one of the most frequently appearing entities on pictorial maps, to the third dimension. Our workflow is based on training data and neural networks for single-view 3D reconstruction of real humans from photos. We first let a network consisting of fully connected layers estimate the depth coordinate of 2D pose points. The gained 3D pose points are inputted together with 2D masks of body parts into a deep implicit surface network to infer 3D signed distance fields (SDFs). By assembling all body parts, we derive 2D depth images and body part masks of the whole figure for different views, which are fed into a fully convolutional network to predict UV images. These UV images and the texture for the given perspective are inserted into a generative network to inpaint the textures for the other views. The textures are enhanced by a cartoonization network and facial details are resynthesized by an autoencoder. Finally, the generated textures are assigned to the inferred body parts in a ray marcher. We test our workflow with 12 pictorial human figures after having validated several network configurations. The created 3D models look generally promising, especially when considering the challenges of silhouette-based 3D recovery and real-time rendering of the implicit SDFs. Further improvement is needed to reduce gaps between the body parts and to add pictorial details to the textures. Overall, the constructed figures may be used for animation and storytelling in digital 3D maps.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
