arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.11529cs.CV
收藏

视频对象分割中的动态查询调制

Video Object Segmentation with Dynamic Query Modulation

Tsinghua University(清华大学);Beijing Institute of Technology(北京理工大学)

Hantao Zhou, Runze Hu, Xiu Li

AI总结:

提出QMVOS动态查询调制方法,将对象特征转化为动态滤波器,通过查询间注意力实现多对象交互,显著提升半监督视频对象分割性能。

摘要:

将中间帧分割结果存储为记忆以进行长程上下文建模,基于时空记忆的方法在半监督视频对象分割(SVOS)中最近展示了令人印象深刻的结果。然而,这些方法面临两个关键限制:1)依赖非局部像素级匹配来读取记忆,导致检索到的特征含有噪声,不利于分割;2)独立分割每个对象,缺乏对象间交互。这些缺点使得基于记忆的方法在相似对象和多对象分割场景中表现不佳。为解决这些问题,我们提出了一种查询调制方法,称为QMVOS。该方法将对象特征概括为动态查询,并将其作为动态滤波器用于掩码预测,从而为模型提供高层描述和对象级感知。通过查询间注意力实现高效且有效的多对象交互。大量实验表明,我们的方法能够为基于记忆的SVOS方法带来显著改进,并在标准SVOS基准上取得有竞争力的性能。代码可在 https://github.com/zht8506/QMVOS 获取。

英文摘要:

Storing intermediate frame segmentations as memory for long-range context modeling, spatial-temporal memory-based methods have recently showcased impressive results in semi-supervised video object segmentation (SVOS). However, these methods face two key limitations: 1) relying on non-local pixel-level matching to read memory, resulting in noisy retrieved features for segmentation; 2) segmenting each object independently without interaction. These shortcomings make the memory-based methods struggle in similar object and multi-object segmentation. To address these issues, we propose a query modulation method, termed QMVOS. This method summarizes object features into dynamic queries and then treats them as dynamic filters for mask prediction, thereby providing high-level descriptions and object-level perception for the model. Efficient and effective multi-object interactions are realized through inter-query attention. Extensive experiments demonstrate that our method can bring significant improvements to the memory-based SVOS method and achieve competitive performance on standard SVOS benchmarks. The code is available at https://github.com/zht8506/QMVOS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑