arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1906.00925cs.CV
收藏

3D外观超分辨率与深度学习

3D Appearance Super-Resolution with Deep Learning

ETH Zurich(苏黎世联邦理工学院);Microsoft(微软)

Yawei Li, Vagia Tsiminaki, Radu Timofte, Marc Pollefeys, Luc van Gool

摘要:

我们解决的问题,检索高分辨率(HR)的对象,从多个视点捕获的纹理图。在多视图的情况下,基于模型的超分辨率(SR)方法最近已被证明恢复高质量的纹理图。另一方面,基于深度学习的方法的出现已经对视频和图像SR的问题产生了重大影响。然而,仍然缺少基于深度学习的方法来超分辨3D对象的外观。在多视图情况下利用深度学习技术的主要限制是缺乏数据。我们基于现有的ETH3D [42],SyB3R [31],MiddleBury以及我们从TUM [21],Fountain [51]和Relief [53]收集的3D场景引入了3D外观SR(3DASR)数据集。我们提供了高分辨率和低分辨率的纹理图,3D几何模型,图像和投影矩阵。我们利用2D学习为基础的SR方法的力量和设计适合于3D多视图的情况下的网络。我们通过引入法向图来融入几何信息,并进一步改进学习过程。实验结果表明,我们提出的网络成功地结合了三维几何信息和超分辨率的纹理映射。

英文摘要:

We tackle the problem of retrieving high-resolution (HR) texture maps of objects that are captured from multiple view points. In the multi-view case, model-based super-resolution (SR) methods have been recently proved to recover high quality texture maps. On the other hand, the advent of deep learning-based methods has already a significant impact on the problem of video and image SR. Yet, a deep learning-based approach to super-resolve the appearance of 3D objects is still missing. The main limitation of exploiting the power of deep learning techniques in the multi-view case is the lack of data. We introduce a 3D appearance SR (3DASR) dataset based on the existing ETH3D [42], SyB3R [31], MiddleBury, and our Collection of 3D scenes from TUM [21], Fountain [51] and Relief [53]. We provide the high- and low-resolution texture maps, the 3D geometric model, images and projection matrices. We exploit the power of 2D learning-based SR methods and design networks suitable for the 3D multi-view case. We incorporate the geometric information by introducing normal maps and further improve the learning process. Experimental results demonstrate that our proposed networks successfully incorporate the 3D geometric information and super-resolve the texture maps.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑