基于无噪声文本到视频扩散模型的时间条件指称视频目标分割
Temporal-Conditional Referring Video Object Segmentation with Noise-Free Text-to-Video Diffusion Model
AI总结：
本文提出一种时间条件RVOS模型，通过改进分割头设计、利用无噪声文本到视频扩散模型提取特征，并设计TCMR模块，在四个公开基准上取得最先进性能。
摘要：
指称视频目标分割（RVOS）旨在根据文本描述分割视频中的特定目标。我们观察到，近年来的RVOS方法往往过度关注特征提取和时间建模，而相对忽视了分割头的设计。事实上，分割头设计仍有相当大的改进空间。为解决这一问题，我们提出了一种时间条件指称视频目标分割模型，该模型创新性地整合现有分割方法，有效增强边界分割能力。此外，我们的模型利用文本到视频扩散模型进行特征提取。在此基础上，我们去除了传统的噪声预测模块，以避免噪声的随机性降低分割精度，从而在简化模型的同时提升性能。最后，为克服VAE有限的特征提取能力，我们设计了时间上下文掩码细化（TCMR）模块，在不引入复杂设计的情况下显著提升分割质量。我们在四个公开RVOS基准上评估了我们的方法，并持续取得最先进的性能。
英文摘要：
Referring Video Object Segmentation (RVOS) aims to segment specific objects in a video according to textual descriptions. We observe that recent RVOS approaches often place excessive emphasis on feature extraction and temporal modeling, while relatively neglecting the design of the segmentation head. In fact, there remains considerable room for improvement in segmentation head design. To address this, we propose a Temporal-Conditional Referring Video Object Segmentation model, which innovatively integrates existing segmentation methods to effectively enhance boundary segmentation capability. Furthermore, our model leverages a text-to-video diffusion model for feature extraction. On top of this, we remove the traditional noise prediction module to avoid the randomness of noise from degrading segmentation accuracy, thereby simplifying the model while improving performance. Finally, to overcome the limited feature extraction capability of the VAE, we design a Temporal Context Mask Refinement (TCMR) module, which significantly improves segmentation quality without introducing complex designs. We evaluate our method on four public RVOS benchmarks, where it consistently achieves state-of-the-art performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
