基于场景的自动驾驶汽车预测模型评价
Scenario-based Evaluation of Prediction Models for Automated Vehicles
摘要:
为了安全运行,自动驾驶汽车(AV)必须预测周围环境的变化。为此,重要的是要知道哪些预测模型最适合每种情况。目前,预测模型的评估通常是在一组轨迹上执行的,而不区分它们捕获的运动类型,导致无法确定每个模型对不同情况的适用性。在这项工作中,我们说明了标准化的评估方法如何导致错误的结论模型的预测能力,防止明确的评估预测模型,并可能导致危险的道路情况。我们认为,以下的评价实践中的安全评估AV,评估的预测模型,应在一个基于神经网络的方式进行。为了鼓励对预测模型进行基于统计的评估,并说明评估不当的危险,我们根据它们捕获的运动类型对Waymo Open Motion数据集的轨迹进行了分类。接下来,针对不同的轨迹类型和预测范围,对三种不同的模型进行了全面评估。结果表明,共同的评价方法是不够的,评估应根据应用程序中的模型将操作。
英文摘要:
To operate safely, an automated vehicle (AV) must anticipate how the environment around it will evolve. For that purpose, it is important to know which prediction models are most appropriate for every situation. Currently, assessment of prediction models is often performed over a set of trajectories without distinction of the type of movement they capture, resulting in the inability to determine the suitability of each model for different situations. In this work we illustrate how standardized evaluation methods result in wrong conclusions regarding a model's predictive capabilities, preventing a clear assessment of prediction models and potentially leading to dangerous on-road situations. We argue that following evaluation practices in safety assessment for AVs, assessment of prediction models should be performed in a scenario-based fashion. To encourage scenario-based assessment of prediction models and illustrate the dangers of improper assessment, we categorize trajectories of the Waymo Open Motion dataset according to the type of movement they capture. Next, three different models are thoroughly evaluated for different trajectory types and prediction horizons. Results show that common evaluation methods are insufficient and the assessment should be performed depending on the application in which the model will operate.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。