你应该正确评分你的里程计

中文摘要 AI 辅助

当我们评估里程计性能时，通常的做法是将估计轨迹与地面真值进行评分。不幸的是，评分使用点度量，如均方根误差，这些度量忽略了滤波器和平滑器等估计器已经报告的协方差矩阵。使用协方差很重要，原因有二。首先，协方差编码了估计器的不确定性，因此它告诉我们估计器是否信任自己的输出。过度自信的估计器不会报告自己迷失。其次，协方差对估计的每个方向上的误差进行加权。没有协方差，估计器会因在不确定方向上的高误差而受到不当惩罚。我们不应使用点度量，而应使用严格适当的评分规则。这些规则将估计与其报告的不确定性一起评分。严格适当的评分规则在没有报告协方差时恢复点度量，并在报告协方差时诊断协方差不一致性。使用单边成对检验，我们表明两个估计器可以在没有地面真值的情况下暴露至少其中一个的过度自信。严格适当的评分规则和我们的成对检验可在我们的开源框架smfeval中获得。作为案例研究，我们使用smfeval评估地面激光雷达-惯性里程计平移分量的不确定性质量。在四个滤波器中，我们发现过度自信——最坏情况报告厘米级确定性但误差达千米级。知道滤波器过度自信后，我们调查其机制。调查将过度自信追溯到滤波器将激光雷达测量归因于比它们实际携带的更多的新信息。

英文摘要

When we evaluate the performance of our odometry, it is common practice to score the estimated track against a ground truth. Unfortunately, scoring uses point metrics, such as the root mean square error, that ignore the covariance matrix which estimators like filters and smoothers already report. Using the covariance matters for two reasons. First, the covariance encodes the estimator's uncertainty, so it tells us whether the estimator trusts its own output. An overconfident estimator will not report itself lost. Second, the covariance weights the error in each direction of the estimate. Without the covariance, an estimator is unduly penalized for a high error in an uncertain direction. Instead of point metrics, we should use strictly proper scoring rules. These rules score the estimate together with its reported uncertainty. Strictly proper scoring rules recover the point metrics when no covariance is reported, and they diagnose covariance inconsistency when covariance is reported. Using a one-sided pairwise test, we show that two estimators can expose overconfidence in at least one of them without a ground truth. Strictly proper scoring rules and our pairwise test are available in our open-source framework smfeval. As a case study, we use smfeval to assess the uncertainty quality of the translational component of ground-based LiDAR-inertial odometry. Across four filters we find overconfidence - the worst case reports centimeter certainty with kilometer error. Knowing the filters are overconfident, we investigate the mechanism. The investigation traces overconfidence to filters crediting LiDAR measurements with more new information than they carry.

发表机构

• IT University of Copenhagen（哥本哈根信息技术大学）

机构由 AI 辅助整理，请以论文原文为准。