arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2205.07058cs.CV
收藏

RTMV:一种光线跟踪的多视图合成数据集

RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesis

NVIDIA(英伟达);University of Maryland College Park(马里兰大学帕克分校)
Jonathan Tremblay, Moustafa Meshry, Alex Evans, Jan Kautz, Alexander Keller, Sameh Khamis, Thomas Müller, Charles Loop, Nathan Morrical, Koki Nagano, Towaki Tak… 展开作者

Jonathan Tremblay, Moustafa Meshry, Alex Evans, Jan Kautz, Alexander Keller, Sameh Khamis, Thomas Müller, Charles Loop, Nathan Morrical, Koki Nagano, Towaki Takikawa, Stan Birchfield

摘要:

我们提出了一个大规模的合成数据集的新的视图合成包括约300 k的图像渲染从近2000个复杂的场景,使用高品质的光线追踪在高分辨率(1600 × 1600像素)。该数据集比现有的合成数据集大几个数量级,从而为训练和评估提供了一个大的统一基准。使用4个不同的高质量3D网格源,我们数据集的场景在相机视图、照明、形状、材料和纹理方面表现出具有挑战性的变化。由于我们的数据集太大,现有的方法处理,我们提出了稀疏体素光场(SVLF),一个有效的基于体素的光场方法,用于新的视图合成,实现了类似的性能NeRF的合成数据,同时是一个数量级快训练和两个数量级快渲染。SVLF通过依赖于稀疏体素八叉树、仔细的体素采样(每条射线只需要少量查询)和简化的网络结构以及训练时的地面真实深度图来实现这种速度。我们的数据集是由NViSII生成的,NViSII是一个基于Python的光线追踪渲染器,它被设计为非专家使用和共享简单,通过使用脚本灵活而强大,并且能够创建高质量和基于物理的渲染图像。使用我们数据集的一个子集进行的实验使我们能够比较标准方法,如NeRF和mip-NeRF用于单场景建模,以及pixelNeRF用于类别级建模,指出未来需要在这一领域进行改进。

英文摘要:

We present a large-scale synthetic dataset for novel view synthesis consisting of ~300k images rendered from nearly 2000 complex scenes using high-quality ray tracing at high resolution (1600 x 1600 pixels). The dataset is orders of magnitude larger than existing synthetic datasets for novel view synthesis, thus providing a large unified benchmark for both training and evaluation. Using 4 distinct sources of high-quality 3D meshes, the scenes of our dataset exhibit challenging variations in camera views, lighting, shape, materials, and textures. Because our dataset is too large for existing methods to process, we propose Sparse Voxel Light Field (SVLF), an efficient voxel-based light field approach for novel view synthesis that achieves comparable performance to NeRF on synthetic data, while being an order of magnitude faster to train and two orders of magnitude faster to render. SVLF achieves this speed by relying on a sparse voxel octree, careful voxel sampling (requiring only a handful of queries per ray), and reduced network structure; as well as ground truth depth maps at training time. Our dataset is generated by NViSII, a Python-based ray tracing renderer, which is designed to be simple for non-experts to use and share, flexible and powerful through its use of scripting, and able to create high-quality and physically-based rendered images. Experiments with a subset of our dataset allow us to compare standard methods like NeRF and mip-NeRF for single-scene modeling, and pixelNeRF for category-level modeling, pointing toward the need for future improvements in this area.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑