arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.16563cs.CVcs.GR
收藏

基于物理的差分深度传感器仿真

Physics-based Differentiable Depth Sensor Simulation

Siemens Technology(西门子科技)

Benjamin Planche, Rajat Vikram Singh

摘要:

基于递归的算法对于现代计算机视觉和图形应用至关重要,可以实现基于学习的优化和逆问题。例如,用于彩色图像的照片级真实感可区分渲染流水线已经被证明对于旨在映射2D和3D域的应用非常有价值。然而,据我们所知,迄今为止还没有努力将这些基于梯度的方法扩展到深度(2.5D)图像的生成,因为模拟结构光深度传感器意味着解决复杂的光传输和立体匹配问题。在本文中,我们介绍了一种新的端到端可区分的仿真管道生成逼真的2.5D扫描,建立在基于物理的3D渲染和自定义块匹配算法。每个模块可以根据传感器和场景参数进行区分;例如,在一些提供的扫描上自动调整新设备的模拟,或在更大的计算机视觉应用中利用管道作为3D到2.5D的转换器Transformer。应用于各种基于深度的识别任务(分类,姿态估计,语义分割)的深度学习方法的训练,我们的模拟大大提高了实际扫描的结果模型的性能,从而证明了其合成深度数据与以前的静态模拟和基于学习的领域自适应方案相比的保真度和价值。

英文摘要:

Gradient-based algorithms are crucial to modern computer-vision and graphics applications, enabling learning-based optimization and inverse problems. For example, photorealistic differentiable rendering pipelines for color images have been proven highly valuable to applications aiming to map 2D and 3D domains. However, to the best of our knowledge, no effort has been made so far towards extending these gradient-based methods to the generation of depth (2.5D) images, as simulating structured-light depth sensors implies solving complex light transport and stereo-matching problems. In this paper, we introduce a novel end-to-end differentiable simulation pipeline for the generation of realistic 2.5D scans, built on physics-based 3D rendering and custom block-matching algorithms. Each module can be differentiated w.r.t sensor and scene parameters; e.g., to automatically tune the simulation for new devices over some provided scans or to leverage the pipeline as a 3D-to-2.5D transformer within larger computer-vision applications. Applied to the training of deep-learning methods for various depth-based recognition tasks (classification, pose estimation, semantic segmentation), our simulation greatly improves the performance of the resulting models on real scans, thereby demonstrating the fidelity and value of its synthetic depth data compared to previous static simulations and learning-based domain adaptation schemes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑