面向自动驾驶的多相机鸟瞰图感知
Multi-camera Bird's Eye View Perception for Autonomous Driving
AI总结：
本文综述了自动驾驶中基于多相机和深度神经网络的鸟瞰图感知方法，分析了其相比传统逆透视映射的优势，并探讨了传感器融合、下游任务集成及当前挑战。
摘要：
大多数自动驾驶系统包含多样化的传感器组合，包括多个相机、雷达和激光雷达，以确保近处和远处区域的完整360°覆盖。与直接在三维空间中测量的雷达和激光雷达不同，相机捕获的是二维透视投影，存在固有的深度歧义。然而，必须在三维空间中生成感知输出，以便对其他智能体和结构进行空间推理，从而实现最优路径规划。三维空间通常通过省略与高度维度对应的、相关性较低的Z坐标，简化为鸟瞰图（BEV）空间。从相机图像获得所需BEV表示的最基本方法是逆透视映射（IPM），该方法假设地面是平坦的。在新车辆中相当常见的环视视觉系统利用IPM原理生成BEV图像，并将其显示给驾驶员。然而，这种方法不适合自动驾驶，因为这种过于简化的变换方法会引入严重的畸变。更新的方法使用深度神经网络直接在BEV空间中输出结果。这些方法在网络中隐式或显式地利用几何约束将相机图像变换到BEV空间。由于卷积神经网络（CNN）具有更多的上下文信息，并且可学习的变换可以更灵活地适应图像内容，基于深度学习的方法为BEV变换设立了新的基准，并实现了最先进的性能。本章首先讨论了基于多相机的深度神经网络（DNN）模型直接在BEV空间中输出目标表示的当代趋势。然后，我们讨论了该方法如何扩展到有效的传感器融合，并与下游任务（如态势分析和预测）耦合。最后，我们展示了BEV感知中的挑战和开放问题。
英文摘要：
Most automated driving systems comprise a diverse sensor set, including several cameras, Radars, and LiDARs, ensuring a complete 360°coverage in near and far regions. Unlike Radar and LiDAR, which measure directly in 3D, cameras capture a 2D perspective projection with inherent depth ambiguity. However, it is essential to produce perception outputs in 3D to enable the spatial reasoning of other agents and structures for optimal path planning. The 3D space is typically simplified to the BEV space by omitting the less relevant Z-coordinate, which corresponds to the height dimension.The most basic approach to achieving the desired BEV representation from a camera image is IPM, assuming a flat ground surface. Surround vision systems that are pretty common in new vehicles use the IPM principle to generate a BEV image and to show it on display to the driver. However, this approach is not suited for autonomous driving since there are severe distortions introduced by this too-simplistic transformation method. More recent approaches use deep neural networks to output directly in BEV space. These methods transform camera images into BEV space using geometric constraints implicitly or explicitly in the network. As CNN has more context information and a learnable transformation can be more flexible and adapt to image content, the deep learning-based methods set the new benchmark for BEV transformation and achieve state-of-the-art performance. First, this chapter discusses the contemporary trends of multi-camera-based DNN (deep neural network) models outputting object representations directly in the BEV space. Then, we discuss how this approach can extend to effective sensor fusion and coupling downstream tasks like situation analysis and prediction. Finally, we show challenges and open problems in BEV perception.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
