LiAM-SAM：面向鲁棒SAM2多目标跟踪的生命周期感知记忆

中文摘要 AI 辅助

基于分割的多目标跟踪（MOT）结合SAM2等基础视频模型，提供了强大的定位质量，但在拥挤的真实场景中仍然脆弱。在检测器提示的SAM2流程中，失败通常出现在对象生命周期的三个阶段：（i）错误或重复的轨迹初始化，（ii）近距离交互期间的记忆漂移，以及（iii）长时间遮挡或重新进入后不可靠的重新识别。这些错误会污染对象记忆并随时间累积，使得长时跟踪不稳定。在本文中，我们将MOT重新定义为生命周期记忆完整性问题。我们提出了LiAM-SAM，一个生命周期感知记忆（LiAM）框架，针对上述三种失败模式分别设计了针对性机制。在轨迹诞生时，为防止错误或重复初始化，我们应用对比轨迹初始化，将每个提示条件化于现有的邻近跟踪实例。为在强交互期间保持记忆完整性，我们引入了基于运动和几何的记忆校正，以解决交互混淆并抑制漂移。为实现消失后的可靠重新识别，我们维护一个自适应上下文记忆，促进多样且可信的参考作为长期身份锚点。最后，相似性感知的空间剪枝可选地在交叉注意力时选择要保留的记忆令牌，以最小的精度损失提高效率。LiAM-SAM是一个模块化、检测器无关、基于SAM2的MOT系统，在评估基准上达到了最先进的HOTA和IDF1。在关联挑战性环境中，我们的消融实验表明，LiAM将检测器+SAM2基线提高了+10.5 HOTA、+17.4 AssA，并将身份切换减少了96%。

英文摘要

Segmentation-based multi-object tracking (MOT) with foundation video models such as SAM2 offers strong localization quality, yet remains fragile in crowded, real-world scenes. In detector-prompted SAM2 pipelines, failures typically arise at three stages of the object lifecycle: (i) erroneous or duplicate track initiation, (ii) memory drift during close interactions, and (iii) unreliable re-identification after long occlusions or re-entry. These errors corrupt object memory and accumulate over time, making long-horizon tracking unstable. In this paper, we reframe MOT as a lifecycle memory integrity problem. We present LiAM-SAM, a Lifecycle-Aware Memory (LiAM) framework with targeted mechanisms for each of the three failure modes. At track birth, to prevent faulty or duplicate initiations, we apply contrastive track initiation, which conditions each prompt on existing nearby tracked instances. To preserve memory integrity during strong interactions, we introduce motion- and geometry-grounded memory correction that resolves interaction confusions and suppresses drift. For reliable re-identification after disappearance, we maintain an adaptive context memory that promotes diverse and trustworthy references as long-term identity anchors. Finally, similarity aware spatial pruning optionally selects the memory tokens to retain at cross-attention time, improving efficiency with minimal accuracy loss. LiAM-SAM represents a modular, detector-agnostic, SAM2-based MOT system that achieves state-of-the-art HOTA and IDF1 on the evaluated benchmarks. In association-challenging environments, our ablations show that LiAM improves a detector+SAM2 baseline by +10.5 HOTA, +17.4 AssA, and reduces identity switches by 96%.

发表机构

• Toyota Motor Europe（丰田汽车欧洲公司）

机构由 AI 辅助整理，请以论文原文为准。