arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.16256cs.CVcs.AI
收藏

DL3DV-10K:面向基于深度学习的3D视觉的大规模场景数据集

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision

Purdue University(普渡大学);Adobe Inc.(Adobe公司);Rutgers University(罗格斯大学);Google Inc.(谷歌公司);Huazhong University of Science and Technology(华中科技大学);Wormpex AI Research(Wormpex AI研究院)
Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, Xuanmao Li, Xingpeng Sun, Rohan Ashok, Aniruddha Muk… 展开作者

Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, Xuanmao Li, Xingpeng Sun, Rohan Ashok, Aniruddha Mukherjee, Hao Kang, Xiangrui Kong, Gang Hua, Tianyi Zhang, Bedrich Benes, Aniket Bera

AI总结:

针对现有3D视觉数据集规模不足的问题,提出包含5120万帧、10510个视频的大规模真实场景数据集DL3DV-10K,并在其上全面基准测试了NVS方法,验证了数据集对学习可泛化NeRF及3D基础模型的关键价值。

摘要:

我们见证了基于深度学习的3D视觉领域的显著进展,涵盖从基于神经辐射场(NeRF)的3D表示学习到新视角合成(NVS)的应用。然而,现有的基于深度学习的3D视觉场景级数据集要么局限于合成环境,要么仅包含少量真实世界场景,相当匮乏。这种匮乏不仅阻碍了对现有方法的全面基准测试,也限制了基于深度学习的3D分析所能探索的边界。为填补这一关键空白,我们提出了DL3DV-10K,一个大规模场景数据集,包含来自65类兴趣点(POI)位置拍摄的10,510个视频的5120万帧,涵盖有界和无界场景,以及不同程度的反射、透明度和光照。我们在DL3DV-10K上对近期NVS方法进行了全面基准测试,为未来NVS研究揭示了有价值的见解。此外,我们在从DL3DV-10K学习可泛化NeRF的试点研究中取得了令人鼓舞的结果,这表明了大规模场景级数据集的必要性,以期为学习3D表示的基础模型铺平道路。我们的DL3DV-10K数据集、基准测试结果和模型将在 https://dl3dv-10k.github.io/DL3DV-10K/ 公开访问。

英文摘要:

We have witnessed significant progress in deep learning-based 3D vision, ranging from neural radiance field (NeRF) based 3D representation learning to applications in novel view synthesis (NVS). However, existing scene-level datasets for deep learning-based 3D vision, limited to either synthetic environments or a narrow selection of real-world scenes, are quite insufficient. This insufficiency not only hinders a comprehensive benchmark of existing methods but also caps what could be explored in deep learning-based 3D analysis. To address this critical gap, we present DL3DV-10K, a large-scale scene dataset, featuring 51.2 million frames from 10,510 videos captured from 65 types of point-of-interest (POI) locations, covering both bounded and unbounded scenes, with different levels of reflection, transparency, and lighting. We conducted a comprehensive benchmark of recent NVS methods on DL3DV-10K, which revealed valuable insights for future research in NVS. In addition, we have obtained encouraging results in a pilot study to learn generalizable NeRF from DL3DV-10K, which manifests the necessity of a large-scale scene-level dataset to forge a path toward a foundation model for learning 3D representation. Our DL3DV-10K dataset, benchmark results, and models will be publicly accessible at https://dl3dv-10k.github.io/DL3DV-10K/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑