Omnidata:一个可扩展的管道,用于从3D扫描生成多任务中级视觉数据集
Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans
摘要:
本文介绍了一种管道,用于从真实世界的全面3D扫描中参数化采样和渲染多任务视觉数据集。改变采样参数允许人们“操纵”生成的数据集以强调特定信息。除了能够进行有趣的研究之外,我们还展示了工具和生成的数据足以训练鲁棒的视觉模型。在生成的初始数据集上训练的通用架构在多个通用视觉任务和基准测试中达到了最先进的性能,尽管没有看到基准测试或非管道数据。深度估计网络的性能优于MiDaS,表面法线估计网络是第一个在野外表面法线估计方面实现人类水平性能的网络-至少根据OASIS基准的一个指标。带有CLI的Dockerized管道,(主要是Python)代码,用于生成数据的PyTorch数据加载器,生成的启动数据集,下载脚本和其他实用程序可通过我们的项目网站https://omnidata.vision获得。
英文摘要:
This paper introduces a pipeline to parametrically sample and render multi-task vision datasets from comprehensive 3D scans from the real world. Changing the sampling parameters allows one to "steer" the generated datasets to emphasize specific information. In addition to enabling interesting lines of research, we show the tooling and generated data suffice to train robust vision models. Common architectures trained on a generated starter dataset reached state-of-the-art performance on multiple common vision tasks and benchmarks, despite having seen no benchmark or non-pipeline data. The depth estimation network outperforms MiDaS and the surface normal estimation network is the first to achieve human-level performance for in-the-wild surface normal estimation -- at least according to one metric on the OASIS benchmark. The Dockerized pipeline with CLI, the (mostly python) code, PyTorch dataloaders for the generated data, the generated starter dataset, download scripts and other utilities are available through our project website, https://omnidata.vision.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。