arXiv 2307.088502024-11-18cs.CVcs.RO

LiDAR-BEVMTN：面向自动驾驶的实时LiDAR鸟瞰图多任务感知网络

LiDAR-BEVMTN: Real-Time LiDAR Bird's-Eye View Multi-Task Perception Network for Autonomous Driving

Valeo（法雷奥）；TU Ilmenau（伊尔梅瑙工业大学）

Sambit Mohapatra, Senthil Yogamani, Varun Ravi Kumar, Stefan Milz, Heinrich Gotzig, Patrick Mäder

备注：: Accepted for publication at IEEE Transactions on Intelligent Transportation Systems

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出LiDAR-BEVMTN，一种基于共享编码器和任务特定解码器的实时多任务卷积网络，通过SWAG模块和异构训练统一实现LiDAR点云的检测、语义与运动分割，在嵌入式平台以3ms延迟取得领先性能。

摘要：

LiDAR对于自动驾驶中鲁棒的3D场景感知至关重要。继相机感知之后，LiDAR感知拥有最多的研究文献。然而，使用LiDAR进行检测、分割和运动估计等多任务学习的研究仍然相对较少，尤其是在汽车级嵌入式平台上。我们提出了一种用于基于LiDAR的目标检测、语义分割和运动分割的实时多任务卷积神经网络。该统一架构包含一个共享编码器和任务特定解码器，能够实现联合表示学习。我们提出了一种新颖的语义加权与引导（SWAG）模块，用于选择性地传递语义特征以改进目标检测。我们的异构训练方案结合了多样化的数据集，并利用任务之间的互补线索。该工作首次在嵌入式平台上统一实现了这些基于LiDAR点云的关键感知任务，在嵌入式NVIDIA Xavier平台上实现了3毫秒的延迟。我们在语义分割和运动分割两项任务上取得了最先进的结果，在3D目标检测上取得了接近最先进的性能。通过最大化硬件效率并利用多任务协同效应，我们的方法为真实世界的自动驾驶部署提供了一种准确且高效的解决方案。定性结果可见于https://youtu.be/H-hWRzv2lIY。

英文摘要：

LiDAR is crucial for robust 3D scene perception in autonomous driving. LiDAR perception has the largest body of literature after camera perception. However, multi-task learning across tasks like detection, segmentation, and motion estimation using LiDAR remains relatively unexplored, especially on automotive-grade embedded platforms. We present a real-time multi-task convolutional neural network for LiDAR-based object detection, semantics, and motion segmentation. The unified architecture comprises a shared encoder and task-specific decoders, enabling joint representation learning. We propose a novel Semantic Weighting and Guidance (SWAG) module to transfer semantic features for improved object detection selectively. Our heterogeneous training scheme combines diverse datasets and exploits complementary cues between tasks. The work provides the first embedded implementation unifying these key perception tasks from LiDAR point clouds achieving 3ms latency on the embedded NVIDIA Xavier platform. We achieve state-of-the-art results for two tasks, semantic and motion segmentation, and close to state-of-the-art performance for 3D object detection. By maximizing hardware efficiency and leveraging multi-task synergies, our method delivers an accurate and efficient solution tailored for real-world automated driving deployment. Qualitative results can be seen at https://youtu.be/H-hWRzv2lIY.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。