解锁表达中的运动：用于指称视频目标分割的时间校准
Unlocking Motion in Expressions: Temporal Calibration for Referring Video Object Segmentation
浏览论文内容
中文总结 AI 辅助
针对指称视频目标分割中运动语义依赖未显式建模的问题，提出EMC框架，通过MSP、MIC、STSC模块校准运动线索，在6个基准上验证了方法的优越性。
中文摘要 AI 辅助
指称视频目标分割（RVOS）旨在基于自然语言描述，在视频序列中对被指称目标进行像素级分割。现有方法通常在统一的跨模态时间建模框架内引入运动信息，其中语言线索用于目标定位与分割，但未显式建模表达对运动语义的依赖，导致难以根据不同语义需求自适应调整运动信息的使用。为解决这些问题，我们提出用于RVOS的表达驱动运动校准（EMC）框架，该框架显式解锁并利用表达中的运动语义。所提方法通过运动信号处理（MSP）模块从表达中提取可解释的运动控制信号，并采用运动影响校准（MIC）模块在时间决策过程中调整运动线索的贡献；此外，引入语义时间阶段构建（STSC）模块以构建与表达相关的时间阶段，为运动校准提供紧凑的时间候选空间。通过在Ref-YouTubeVOS、Ref-DAVIS17、MeViS（valid/valid$^u$）、A2D-Sentences和JHMDB-Sentences共6个标准基准上进行广泛评估，验证了所提方法的优越性。我们将在该httpsURL上发布代码。
英文摘要
Referring Video Object Segmentation (RVOS) aims to segment referred objects at the pixel level in video sequences based on natural language descriptions. Existing methods typically introduce motion information within a unified cross-modal temporal modeling framework, where language cues are used for target localization and segmentation. However, the dependency of expressions on motion semantics is not explicitly modeled, making it difficult to adaptively adjust the use of motion information according to different semantic requirements. To address these issues, we propose an Expression-driven Motion Calibration (EMC) framework for RVOS that explicitly unlocks and leverages the motion semantics within expressions. The proposed method extracts interpretable motion control signals from expressions via a Motion Signal Processing (MSP) module, and employs a Motion Influence Calibration (MIC) module to adjust the contribution of motion cues during temporal decision making. In addition, a Semantic Temporal Stage Construction (STSC) module is introduced to build expression-relevant temporal stages, providing a compact temporal candidate space for motion calibration. Through extensive evaluation on six standard benchmarks, including Ref-YouTubeVOS, Ref-DAVIS17, MeViS (valid/valid$^u$), A2D-Sentences, and JHMDB-Sentences, the superiority of our method is validated. We will release the code on https://github.com/Jeven7/EMC.
