StreamDAM：面向实时流视频目标分割的存在感知内存
StreamDAM: Presence-Aware Memory for Real-Time Streaming Video Object Segmentation
浏览论文内容
中文总结 AI 辅助
本文针对实时流视频目标分割中现有离线模型无法适配帧率约束、内存机制失效的问题，提出StreamDAM方法，通过模型内优化和存在信号控制，实现了最优的流跟踪性能，恢复了离线模型的大部分准确率。
中文摘要 AI 辅助
DAM4SAM这类质量级视频目标分割（VOS）跟踪器在准确率排行榜上位居前列，但它们是离线运行的，一次处理一帧且无时间限制。在30帧每秒的诚实流协议下，若某一帧超出预算，系统会提供已计算出的最后一个掩码，此时原本的最优方法会失效：支撑其准确率的丰富内存运行速度过慢，无法跟上节奏，且输出结果无法感知目标是否存在。我们将这两个问题追溯至跟踪器的内存流水线，并针对流场景对其进行重构。\nStreamDAM（即本文提出的方法）通过模型内优化而非附加的回退机制，使内存机制本身以帧速率运行，同时用单个学习到的存在信号对其进行控制，该信号决定哪些内容进入内存、跟踪器读取的回溯范围、何时暂不输出以及何时重新检测。机制分析表明，固定策略无法取得最优效果：当目标真正消失时有效的控制，在目标只是难以被观测到时反而会产生负面影响，因此必须逐帧做出选择。在四个基准测试和五个现代基线方法上，StreamDAM是表现最优的流跟踪器，在时间约束下恢复了离线模型几乎所有的准确率，且在最难的内容上，其性能超过了它所基于的离线模型。
英文摘要
Quality-tier video object segmentation (VOS) trackers such as DAM4SAM top accuracy leaderboards, but they are measured offline, one frame at a time with no clock. Under an honest streaming protocol at 30 frames per second, where a frame that misses its budget is served the last mask already computed, the winner collapses: the rich memory that makes it accurate is too slow to keep up, and what it emits is blind to whether the object is even present. We trace both failures to one place, the tracker's memory pipeline, and rebuild it for streaming. \method{} makes the memory machinery itself run at frame rate through in-model optimization rather than a bolted-on fallback, and governs it with a single learned presence signal that decides what enters memory, how far back the tracker reads, when to withhold output, and when to re-detect. A mechanism analysis shows why a fixed policy cannot win: the control that helps when an object truly disappears is the one that hurts when it is merely hard to see, so the choice must be made per frame. Across four benchmarks and five modern baselines, \method{} is the strongest streaming tracker, recovers nearly all of the offline model's accuracy under the clock, and on the hardest content exceeds the offline model it is built from.
