arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.02112cs.CV
收藏

S-NeRF++：基于神经重建与生成的自动驾驶仿真

S-NeRF++: Autonomous Driving Simulation via Neural Reconstruction and Generation

School of Data Science, Fudan University（复旦大学数据科学学院）；University of Oxford（牛津大学）

Yurui Chen, Junge Zhang, Ziyang Xie, Wenye Li, Feihu Zhang, Jiachen Lu, Li Zhang

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出基于神经重建的自动驾驶仿真系统S-NeRF++，通过改进场景参数化与相机位姿学习、利用稀疏LiDAR优化深度，并融合光照阴影效果，实现了大规模场景与移动车辆的高质量重建与仿真，有效提升了下游感知任务性能。

摘要：

自动驾驶仿真系统在增强自动驾驶数据和模拟复杂罕见的交通场景以确保导航安全方面发挥着至关重要的作用。然而，传统的仿真系统通常严重依赖手工建模和2D图像编辑，难以扩展到广阔的场景并生成逼真的仿真数据。在本研究中，我们提出了S-NeRF++，一种基于神经重建的创新型自动驾驶仿真系统。S-NeRF++在nuScenes和Waymo等广泛使用的自动驾驶数据集上进行训练，能够生成大量逼真的街景和前景物体，具有高渲染质量，并在操作和仿真方面提供极大的灵活性。具体而言，S-NeRF++是一种用于合成大规模场景和移动车辆的增强型神经辐射场，改进了场景参数化和相机位姿学习。该系统有效利用含噪且稀疏的LiDAR数据来优化训练并处理深度异常值，确保高质量的重建和新视角渲染。它还通过重建和生成不同的前景车辆，提供了丰富的前景资产库，以支持全面的场景创建。此外，我们开发了一种先进的前景-背景融合流水线，巧妙地融合了光照和阴影效果，进一步增强了仿真的真实感。借助S-NeRF++提供的高质量仿真数据，我们发现感知方法在多个自动驾驶下游任务上获得了性能提升，进一步证明了我们提出的仿真器的有效性。

英文摘要：

Autonomous driving simulation system plays a crucial role in enhancing self-driving data and simulating complex and rare traffic scenarios, ensuring navigation safety. However, traditional simulation systems, which often heavily rely on manual modeling and 2D image editing, struggled with scaling to extensive scenes and generating realistic simulation data. In this study, we present S-NeRF++, an innovative autonomous driving simulation system based on neural reconstruction. Trained on widely-used self-driving datasets such as nuScenes and Waymo, S-NeRF++ can generate a large number of realistic street scenes and foreground objects with high rendering quality as well as offering considerable flexibility in manipulation and simulation. Specifically, S-NeRF++ is an enhanced neural radiance field for synthesizing large-scale scenes and moving vehicles, with improved scene parameterization and camera pose learning. The system effectively utilizes noisy and sparse LiDAR data to refine training and address depth outliers, ensuring high-quality reconstruction and novel-view rendering. It also provides a diverse foreground asset bank by reconstructing and generating different foreground vehicles to support comprehensive scenario creation.Moreover, we have developed an advanced foreground-background fusion pipeline that skillfully integrates illumination and shadow effects, further enhancing the realism of our simulations. With the high-quality simulated data provided by our S-NeRF++, we found the perception methods enjoy performance boosts on several autonomous driving downstream tasks, further demonstrating our proposed simulator's effectiveness.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑