arXiv 2510.061392026-02-26cs.CV

视频变形到掩码：用于指认视频分割的流匹配

Deforming Videos to Masks: Flow Matching for Referring Video Segmentation

SGIT AI Lab, State Grid Corporation of China（国网信通研究院）；University of California, San Diego（加州大学圣地亚哥分校）；The Hong Kong University of Science and Technology（香港科技大学）；The University of Tokyo（东京大学）；University of Cambridge（剑桥大学）；Zhejiang University of Technology（浙江工业大学）；Baidu（百度）

Zanyi Wang, Dengyang Jiang, Liuzhuozheng Li, Sizhe Dang, Chengzu Li, Harry Yang, Guang Dai, Mengmeng Wang, Jingdong Wang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出FlowRVS框架，将指认视频分割视为条件连续流问题，通过学习视频整体表示到目标掩码的直接语言引导变形，在多个基准上取得新突破。

摘要：

指认视频对象分割（RVOS）需要根据自然语言描述来分割视频中的特定对象。RVOS的核心挑战是将抽象的语言概念锚定在特定像素集上，并通过视频复杂的动态过程持续分割这些对象。面对这一困难，先前的工作往往将任务分解为一个务实的'定位-然后分割'流程。然而，这种级联设计通过将语义简化为粗粒度的几何提示（例如点）创建了信息瓶颈，并且在分割过程与初始语言定位解耦时难以维持时间一致性。为克服这些根本限制，我们提出了FlowRVS，一种新的框架，将RVOS重新概念化为一个条件连续流问题。这使我们能够利用预训练的T2V模型固有的优势，即细粒度像素控制、文本-视频语义对齐和时间一致性。与传统的从噪声生成到掩码或直接预测掩码不同，我们通过学习从视频整体表示到目标掩码的直接、语言引导的变形来重新制定任务。我们的单阶段生成方法在所有主要RVOS基准上取得了新的最新结果。具体来说，在MeViS上达到51.1的J&F（比先前SOTA高1.6），在零样本Ref-DAVIS17上达到73.3（比先前高2.7），证明了将视频理解任务建模为连续变形过程的显著潜力。

英文摘要：

Referring Video Object Segmentation (RVOS) requires segmenting specific objects in a video guided by a natural language description. The core challenge of RVOS is to anchor abstract linguistic concepts onto a specific set of pixels and continuously segment them through the complex dynamics of a video. Faced with this difficulty, prior work has often decomposed the task into a pragmatic `locate-then-segment' pipeline. However, this cascaded design creates an information bottleneck by simplifying semantics into coarse geometric prompts (e.g, point), and struggles to maintain temporal consistency as the segmenting process is often decoupled from the initial language grounding. To overcome these fundamental limitations, we propose FlowRVS, a novel framework that reconceptualizes RVOS as a conditional continuous flow problem. This allows us to harness the inherent strengths of pretrained T2V models, fine-grained pixel control, text-video semantic alignment, and temporal coherence. Instead of conventional generating from noise to mask or directly predicting mask, we reformulate the task by learning a direct, language-guided deformation from a video's holistic representation to its target mask. Our one-stage, generative approach achieves new state-of-the-art results across all major RVOS benchmarks. Specifically, achieving a J&F of 51.1 in MeViS (+1.6 over prior SOTA) and 73.3 in the zero shot Ref-DAVIS17 (+2.7), demonstrating the significant potential of modeling video understanding tasks as continuous deformation processes.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。