arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.07601cs.CV
收藏

ST-P3:通过时空特征学习实现端到端的基于视觉的自动驾驶

ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning

Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);The University of California, San Diego(加利福尼亚大学圣迭戈分校);JD Explore Academy, JD.com Inc.(京东集团京东探索研究院)

Shengchao Hu, Li Chen, Penghao Wu, Hongyang Li, Junchi Yan, Dacheng Tao

摘要:

许多现有的自动驾驶范例涉及多阶段离散的任务管道。为了更好地预测控制信号并增强用户安全性,需要一种受益于联合时空特征学习的端到端方法。虽然有一些基于激光雷达的输入或隐式设计的开创性工作,在本文中,我们制定了一个可解释的基于视觉的设置的问题。特别是,我们提出了一个时空特征学习方案,同时面向一组更有代表性的感知,预测和规划任务的功能,这是所谓的ST-P3。具体而言,自我中心对齐的积累技术,提出了保留在3D空间的几何信息之前的鸟瞰图变换的感知;双路径建模设计考虑到过去的运动变化,为未来的预测;一个基于时间的细化单元,以补偿识别视觉为基础的元素规划。据我们所知,我们是第一个系统地研究可解释的端到端基于视觉的自动驾驶系统的每个部分的公司。我们对我们的方法进行了基准测试,针对开环nuScenes数据集和闭环CARLA模拟的最新技术。实验结果表明了该方法的有效性。源代码、模型和协议详细信息可在https://github.com/OpenPerceptionX/ST-P3上公开获取。

英文摘要:

Many existing autonomous driving paradigms involve a multi-stage discrete pipeline of tasks. To better predict the control signals and enhance user safety, an end-to-end approach that benefits from joint spatial-temporal feature learning is desirable. While there are some pioneering works on LiDAR-based input or implicit design, in this paper we formulate the problem in an interpretable vision-based setting. In particular, we propose a spatial-temporal feature learning scheme towards a set of more representative features for perception, prediction and planning tasks simultaneously, which is called ST-P3. Specifically, an egocentric-aligned accumulation technique is proposed to preserve geometry information in 3D space before the bird's eye view transformation for perception; a dual pathway modeling is devised to take past motion variations into account for future prediction; a temporal-based refinement unit is introduced to compensate for recognizing vision-based elements for planning. To the best of our knowledge, we are the first to systematically investigate each part of an interpretable end-to-end vision-based autonomous driving system. We benchmark our approach against previous state-of-the-arts on both open-loop nuScenes dataset as well as closed-loop CARLA simulation. The results show the effectiveness of our method. Source code, model and protocol details are made publicly available at https://github.com/OpenPerceptionX/ST-P3.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑