arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.10880cs.CV
收藏

自动驾驶中的开放三维世界

Open 3D World in Autonomous Driving

Beijing University of Posts and Telecommunications(北京邮电大学);University of Washington(华盛顿大学);University of Copenhagen(哥本哈根大学)

Xinlong Cheng, Lei Li

AI总结:

本文提出一种融合激光雷达点云与文本特征的新框架,在自动驾驶中实现开放词汇三维目标检测,并在NuScenes-T和Lyft Level 5数据集上验证了其有效性。

摘要:

开放词汇感知能力代表了自动驾驶系统的一项重大进步,它促进了实时理解和解释各种文本输入的能力。尽管在二维计算机视觉领域对开放词汇任务进行了广泛研究,但将这些方法应用于三维环境,特别是在大规模户外场景中,仍然相对不成熟。本文提出了一种新颖的方法,将来自激光雷达传感器的三维点云数据与文本信息相结合。主要关注点在于利用文本数据直接在自动驾驶环境中定位和识别物体。我们引入了一个高效的框架,用于融合鸟瞰图(BEV)区域特征与文本特征,从而使系统能够无缝适应新的文本输入,并增强开放词汇检测任务的鲁棒性。通过在新增的NuScenes-T数据集上进行的大量实验,对所提方法的有效性进行了严格评估,并在Lyft Level 5数据集上对其零样本性能进行了额外验证。这项研究通过利用多模态数据增强三维环境中的开放词汇感知,为自动驾驶技术的进步做出了实质性贡献,从而推动了自动驾驶导航和感知领域可实现目标的边界。

英文摘要:

The capability for open vocabulary perception represents a significant advancement in autonomous driving systems, facilitating the comprehension and interpretation of a wide array of textual inputs in real-time. Despite extensive research in open vocabulary tasks within 2D computer vision, the application of such methodologies to 3D environments, particularly within large-scale outdoor contexts, remains relatively underdeveloped. This paper presents a novel approach that integrates 3D point cloud data, acquired from LIDAR sensors, with textual information. The primary focus is on the utilization of textual data to directly localize and identify objects within the autonomous driving context. We introduce an efficient framework for the fusion of bird's-eye view (BEV) region features with textual features, thereby enabling the system to seamlessly adapt to novel textual inputs and enhancing the robustness of open vocabulary detection tasks. The effectiveness of the proposed methodology is rigorously evaluated through extensive experimentation on the newly introduced NuScenes-T dataset, with additional validation of its zero-shot performance on the Lyft Level 5 dataset. This research makes a substantive contribution to the advancement of autonomous driving technologies by leveraging multimodal data to enhance open vocabulary perception in 3D environments, thereby pushing the boundaries of what is achievable in autonomous navigation and perception.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑