CRITERIA：一种用于评估自动驾驶轨迹预测模型的新型基准测试范式
CRITERIA: a New Benchmarking Paradigm for Evaluating Trajectory Prediction Models for Autonomous Driving
AI总结：
针对现有自动驾驶轨迹预测基准存在场景偏差、指标有局限的问题，提出CRITERIA新型基准范式，通过细粒度场景提取、无偏差指标及大量实验，实现更准确的模型排名与行为表征。
摘要：
基准测试是评估自动驾驶轨迹预测模型的常用方法。现有基准依赖于偏向巡航等更常见场景的数据集，以及在所有场景上取平均值计算的基于距离的指标。遵循这样的体系，人们几乎无法深入了解模型的特性，包括它们处理不同场景的能力，以及输出的可接受性和多样性。目前已有许多旨在衡量轨迹可接受性和多样性的补充指标，但它们存在诸如轨迹长度之类的偏差。\n在本文中，我们提出了一种用于评估轨迹预测方法的新型基准测试范式（CRITERIA）。具体而言，我们提出：1）一种根据道路结构、模型性能和数据属性提取不同特异性水平的驾驶场景的方法，用于对预测模型进行细粒度排名；2）一组全新的无偏差指标，其中多样性指标结合了给定场景的特征，可接受性指标则受真实驾驶约束的启发，考虑了道路结构和运动学合规性；3）我们使用所提出的基准，借助大规模Argoverse数据集对一组具有代表性的预测模型开展了大量实验。结果表明，所提出的基准能够生成更准确的模型排名，还可作为表征模型行为的手段。我们进一步开展了消融研究，以明确用于计算所提指标的不同要素的贡献。
英文摘要：
Benchmarking is a common method for evaluating trajectory prediction models for autonomous driving. Existing benchmarks rely on datasets, which are biased towards more common scenarios, such as cruising, and distance-based metrics that are computed by averaging over all scenarios. Following such a regiment provides a little insight into the properties of the models both in terms of how well they can handle different scenarios and how admissible and diverse their outputs are. There exist a number of complementary metrics designed to measure the admissibility and diversity of trajectories, however, they suffer from biases, such as length of trajectories. In this paper, we propose a new benChmarking paRadIgm for evaluaTing trajEctoRy predIction Approaches (CRITERIA). Particularly, we propose 1) a method for extracting driving scenarios at varying levels of specificity according to the structure of the roads, models' performance, and data properties for fine-grained ranking of prediction models; 2) A set of new bias-free metrics for measuring diversity, by incorporating the characteristics of a given scenario, and admissibility, by considering the structure of roads and kinematic compliancy, motivated by real-world driving constraints. 3) Using the proposed benchmark, we conduct extensive experimentation on a representative set of the prediction models using the large scale Argoverse dataset. We show that the proposed benchmark can produce a more accurate ranking of the models and serve as a means of characterizing their behavior. We further present ablation studies to highlight contributions of different elements that are used to compute the proposed metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
