SteerSeg: 基于注意力引导的视频分割
SteerSeg: Attention Steering for Reasoning Video Segmentation
AI总结：
SteerSeg通过输入级条件引导解决注意力对齐问题，结合可学习软提示和推理引导的CoT提示，提升视频分割的时空定位能力。
摘要：
视频推理分割需要从自然语言表达中跨视频帧定位对象，通常涉及空间推理和隐含引用。近期方法通过提取冻结的大视觉-语言模型（LVLM）的注意力图作为空间先验进行分割，实现无训练接地。然而，这些注意力图优化于文本生成而非空间定位，常导致模糊和歧义的接地信号。本文提出SteerSeg，一个轻量框架，将注意力对齐问题视为注意力接地的关键瓶颈，并通过输入级条件引导调整注意力源。SteerSeg结合可学习软提示与推理引导的链式推理（CoT）提示。软提示重塑注意力分布以生成更空间集中的图，而CoT衍生属性通过引导注意力至正确实例解决相似对象的歧义。生成的注意力图转换为关键帧点提示以指导分割模型，同时使用基于相关性的评分对候选轨迹进行排名和选择。我们的方法冻结LVLM和分割模型参数，仅学习少量软提示，保留模型的预训练推理能力并显著提升接地能力。尽管仅在Ref-YouTube-VOS上训练，SteerSeg在多样化基准上表现良好，显著提升LVLM的空间接地能力。项目页面：https://steerseg.github.io
英文摘要：
Video reasoning segmentation requires localizing objects across video frames from natural language expressions, often involving spatial reasoning and implicit references. Recent approaches leverage frozen large vision-language models (LVLMs) by extracting attention maps and using them as spatial priors for segmentation, enabling training-free grounding. However, these attention maps are optimized for text generation rather than spatial localization, often resulting in diffuse and ambiguous grounding signals. In this work, we introduce SteerSeg, a lightweight framework that identifies attention misalignment as the key bottleneck in attention-based grounding and proposes to steer attention at its source through input-level conditioning. SteerSeg combines learnable soft prompts with reasoning-guided Chain-of-Thought (CoT) prompting. The soft prompts reshape the attention distribution to produce more spatially concentrated maps, while CoT-derived attributes resolve ambiguity among similar objects by guiding attention toward the correct instance. The resulting attention maps are converted into point prompts across keyframes to guide a segmentation model, while candidate tracklets are ranked and selected using correlation-based scoring. Our approach freezes the LVLM and segmentation model parameters and learns only a small set of soft prompts, preserving the model's pretrained reasoning capabilities while significantly improving grounding. Despite being trained only on Ref-YouTube-VOS, SteerSeg generalizes well across diverse benchmarks, significantly improving the spatial grounding capability of LVLMs. Project page: https://steerseg.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
