arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1809.04843cs.CV
收藏

基于视觉的驾驶模型离线评价研究

On Offline Evaluation of Vision-based Driving Models

Computer Vision Center, Universitat Autònoma de Barcelona(巴塞罗那自治大学计算机视觉中心);Intel Labs(英特尔实验室)

Felipe Codevilla, Antonio M. López, Vladlen Koltun, Alexey Dosovitskiy

摘要:

理想情况下,自动驾驶模型应该通过将其部署在现实世界中的物理车辆上进行评估。不幸的是,这种方法对绝大多数研究人员来说并不实用。一个有吸引力的替代方案是离线评估模型,在预先收集的验证数据集上进行地面实况注释。在本文中,我们研究了各种在线和离线指标之间的关系,用于评估自动驾驶模型。我们发现离线预测误差与驾驶质量不一定相关,预测误差相同的两个模型的驾驶性能可能会有显着差异。我们发现,离线评价与驾驶质量的相关性可以通过选择适当的验证数据集和合适的离线指标得到显着改善。补充视频可以在https://www.youtube.com/watch?上观看v=P8K8Z-iF0cY

英文摘要:

Autonomous driving models should ideally be evaluated by deploying them on a fleet of physical vehicles in the real world. Unfortunately, this approach is not practical for the vast majority of researchers. An attractive alternative is to evaluate models offline, on a pre-collected validation dataset with ground truth annotation. In this paper, we investigate the relation between various online and offline metrics for evaluation of autonomous driving models. We find that offline prediction error is not necessarily correlated with driving quality, and two models with identical prediction error can differ dramatically in their driving performance. We show that the correlation of offline evaluation with driving quality can be significantly improved by selecting an appropriate validation dataset and suitable offline metrics. The supplementary video can be viewed at https://www.youtube.com/watch?v=P8K8Z-iF0cY

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑