ReferDINO：基于视觉定位基础的指代视频目标分割
ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundations
AI总结：
提出ReferDINO模型，通过定位引导可变形掩码解码器和对象一致时间增强器，结合视觉定位基础实现像素级感知与时空推理，在五个基准上取得显著优于以往方法的性能且具备实时推理速度。
摘要：
指代视频目标分割（RVOS）旨在根据文本描述在整个视频中分割目标对象。这极具挑战性，因为它涉及深层的视觉-语言理解、像素级密集预测和时空推理。尽管近年来取得了显著进展，但现有方法在综合考虑所有这些方面时仍存在明显差距。在这项工作中，我们提出了ReferDINO，一个强大的RVOS模型，它从基础视觉定位模型中继承了区域级视觉-语言对齐，并进一步被赋予了像素级密集感知和跨模态时空推理能力。具体而言，ReferDINO集成了两个关键组件：1）定位引导的可变形掩码解码器，利用位置预测通过可微变形机制逐步引导掩码预测；2）对象一致的时间增强器，将预训练的时变文本特征注入帧间交互以捕获对象感知的动态变化。此外，还设计了一种置信度感知的查询剪枝策略，在不损害模型性能的情况下加速对象解码。在五个基准上的广泛实验结果表明，我们的ReferDINO显著优于以前的方法（例如，在Ref-YouTube-VOS上+3.9% (J&F)），并具有实时推理速度（51 FPS）。
英文摘要：
Referring video object segmentation (RVOS) aims to segment target objects throughout a video based on a text description. This is challenging as it involves deep vision-language understanding, pixel-level dense prediction and spatiotemporal reasoning. Despite notable progress in recent years, existing methods still exhibit a noticeable gap when considering all these aspects. In this work, we propose \textbf{ReferDINO}, a strong RVOS model that inherits region-level vision-language alignment from foundational visual grounding models, and is further endowed with pixel-level dense perception and cross-modal spatiotemporal reasoning. In detail, ReferDINO integrates two key components: 1) a grounding-guided deformable mask decoder that utilizes location prediction to progressively guide mask prediction through differentiable deformation mechanisms; 2) an object-consistent temporal enhancer that injects pretrained time-varying text features into inter-frame interaction to capture object-aware dynamic changes. Moreover, a confidence-aware query pruning strategy is designed to accelerate object decoding without compromising model performance. Extensive experimental results on five benchmarks demonstrate that our ReferDINO significantly outperforms previous methods (e.g., +3.9% (\mathcal{J}&\mathcal{F}) on Ref-YouTube-VOS) with real-time inference speed (51 FPS).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
