超越排行榜分数：面向部署的行人中心环境可解释跟踪评估协议
Beyond Leaderboard Scores: A Deployment-Focused Protocol for Interpretable Tracking Evaluation in Pedestrian-Centric Environments
- ETH Zurich（苏黎世联邦理工学院）
- Swiss Paraplegic Research（瑞士截瘫研究中心）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对行人环境中移动机器人跟踪评估，提出仅跟踪器协议，用共享检测隔离跟踪器行为，直接评估关键能力，并在JRDB上验证，发现缺失观测延续是主要限制。
AI中文摘要：
在行人中运行的移动机器人需要轨迹能够快速生成、在观测缺失时保持空间可信度、保持身份一致性，并适应嵌入式计算预算。聚合跟踪分数对轨迹何时以及如何失败提供的洞察有限，而不同的检测器输入可能混淆跟踪器和检测器的质量。我们提出了一种面向部署的、仅跟踪器的评估协议，该协议使用共享检测来隔离跟踪器行为，并直接评估初始化、检测器间隙延续、身份恢复、近邻关联以及负载相关的跟踪器步进运行时间，同时保留高阶跟踪精度（HOTA）作为补充的聚合度量。我们将该协议应用于JackRabbot数据集和基准（JRDB），使用六个开源跟踪器和我们轻量级的行人参考跟踪器（PedRefTrack），以及一个GT辅助变体，该变体在理想化关联和运动下估计剩余的跟踪器侧差距。在固定检测下，非GT跟踪器的HOTA仅跨越24.26%-29.67%，但表现出显著不同的能力概况。在没有检测器支持1.0秒后，没有GT辅助的跟踪器在超过一半的合格案例中保持空间正确且身份一致的输出，使得缺失观测延续成为测试属性中的主要限制。近邻失败较小，且主要在最短间隔处增加。在NVIDIA Jetson Orin上的跟踪器步进运行时间是重尾且负载敏感的，导致多个跟踪器在拥挤帧中低于10 Hz实时目标。该协议提供了一种可复现的方式来表征行人中心环境中的跟踪器行为和部署适用性。代码和评估脚本已在此https URL发布。
英文摘要：
Mobile robots operating among pedestrians need trajectories that become available quickly, remain spatially credible through missed observations, preserve identity, and fit within an embedded computing budget. Aggregate tracking scores provide limited insight into when and how trajectories fail, while varying detector inputs can confound tracker and detector quality. We present a deployment-focused, tracker-only evaluation protocol that uses shared detections to isolate tracker behavior and directly evaluates initialization, detector-gap continuation, identity recovery, close-neighbor association, and load-dependent tracker-step runtime, while Higher Order Tracking Accuracy (HOTA) is retained as a complementary aggregate measure. We apply the protocol to the JackRabbot Dataset and Benchmark (JRDB) using six open-source trackers and our lightweight Pedestrian Reference Tracker (PedRefTrack), together with a GT-assisted variant that estimates the remaining tracker-side gap under idealized association and motion. Under fixed detections, the non-GT trackers span only 24.26%-29.67% HOTA yet exhibit markedly different capability profiles. After 1.0 s without detector support, no tracker without GT assistance maintains spatially correct, same-identity output in more than half of eligible cases, making missing-observation continuation the dominant limitation among the tested properties. Close-neighbor failures are smaller and increase mainly at the shortest separations. Tracker-step runtime on an NVIDIA Jetson Orin is heavy-tailed and load-sensitive, causing several trackers to fall below the 10 Hz real-time target in crowded frames. The protocol provides a reproducible way to characterize tracker behavior and deployment suitability in pedestrian-centric environments. Code and evaluation scripts are released at https://github.com/SCAI-Lab/tracker_eval.
