arXiv 2510.073192025-10-08cs.CV

时间提示至关重要：重新思考指称视频目标分割

Temporal Prompting Matters: Rethinking Referring Video Object Segmentation

Graduate Institute of Communication Engineering, National Taiwan University, Taiwan（台湾国立台湾大学通信工程研究所）；NVIDIA

Ci-Siang Lin, Min-Hung Chen, I-Jieh Liu, Chien-Yi Wang, Sifei Liu, Yu-Chiang Frank Wang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Tenet框架，利用现成检测器与跟踪器生成时间提示，并通过提示偏好学习评估提示质量，以高效适应基础分割模型完成指称视频目标分割，在基准上验证了有效性。

摘要：

指称视频目标分割（RVOS）旨在分割视频中由查询语句所指称的目标。现有的大多数方法需要使用密集掩码标注进行端到端训练，这可能计算量大且可扩展性较差。在这项工作中，我们重新思考了RVOS问题，并旨在探究该任务的关键。基于现有的基础分割模型，我们将RVOS任务分解为指称、视频和分割三个因素，并提出了一个时间提示生成与选择（Tenet）框架来处理指称和视频因素，而将分割问题留给基础模型。为了高效地将基于图像的基础分割模型适应于指称视频目标分割，我们利用现成的目标检测器和跟踪器来生成与指称语句相关的时间提示。虽然可以生成高质量的时间提示，但它们难以从置信度分数中轻易识别出来。为解决这一问题，我们提出了提示偏好学习来评估所生成时间提示的质量。通过采用这些提示来指导基于图像的基础分割模型，我们能够为所指称的目标生成高质量的掩码，从而实现模型对指称视频目标分割的高效适应。在RVOS基准上的实验证明了Tenet框架的有效性。

英文摘要：

Referring Video Object Segmentation (RVOS) aims to segment the object referred to by the query sentence in the video. Most existing methods require end-to-end training with dense mask annotations, which could be computation-consuming and less scalable. In this work, we rethink the RVOS problem and aim to investigate the key to this task. Based on existing foundation segmentation models, we decompose the RVOS task into referring, video, and segmentation factors, and propose a Temporal Prompt Generation and Selection (Tenet) framework to address the referring and video factors while leaving the segmentation problem to foundation models. To efficiently adapt image-based foundation segmentation models to referring video object segmentation, we leverage off-the-shelf object detectors and trackers to produce temporal prompts associated with the referring sentence. While high-quality temporal prompts could be produced, they can not be easily identified from confidence scores. To tackle this issue, we propose Prompt Preference Learning to evaluate the quality of the produced temporal prompts. By taking such prompts to instruct image-based foundation segmentation models, we would be able to produce high-quality masks for the referred object, enabling efficient model adaptation to referring video object segmentation. Experiments on RVOS benchmarks demonstrate the effectiveness of the Tenet framework.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。