arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.07922cs.CV
收藏

学习质量感知的动态记忆视频对象分割

Learning Quality-aware Dynamic Memory for Video Object Segmentation

Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院,清华大学);Huawei Technologies(华为技术有限公司);University College London(伦敦大学学院)

Yong Liu, Ran Yu, Fei Yin, Xinyuan Zhao, Wei Zhao, Weihao Xia, Yujiu Yang

摘要:

最近,一些基于时空记忆的方法已经证明,存储中间帧和它们的掩码作为记忆有助于分割视频中的目标对象。然而,它们主要关注当前帧和内存帧之间的更好匹配,而没有明确关注内存的质量。因此,具有较差分割模板的帧容易被记忆,这导致分割模板误差累积问题,并进一步影响分割性能。此外,内存帧数随帧数增长而线性增长,也限制了模型处理长视频的能力。为此,我们提出了一个质量感知的动态内存网络(QDMN)来评估每个帧的分割质量,允许存储体有选择地存储准确分割的帧,以防止错误积累的问题。然后,我们结合分割质量和时间一致性动态更新的存储体,以提高模型的实用性。在没有任何附加功能的情况下,我们的QDMN在DAVIS和YouTube-VOS基准测试中实现了最先进的性能。此外,大量的实验表明,所提出的质量评估模块(QAM)可以应用于基于内存的方法作为通用插件,并显着提高性能。我们的源代码可在https://github.com/workforai/QDMN上获得。

英文摘要:

Recently, several spatial-temporal memory-based methods have verified that storing intermediate frames and their masks as memory are helpful to segment target objects in videos. However, they mainly focus on better matching between the current frame and the memory frames without explicitly paying attention to the quality of the memory. Therefore, frames with poor segmentation masks are prone to be memorized, which leads to a segmentation mask error accumulation problem and further affect the segmentation performance. In addition, the linear increase of memory frames with the growth of frame number also limits the ability of the models to handle long videos. To this end, we propose a Quality-aware Dynamic Memory Network (QDMN) to evaluate the segmentation quality of each frame, allowing the memory bank to selectively store accurately segmented frames to prevent the error accumulation problem. Then, we combine the segmentation quality with temporal consistency to dynamically update the memory bank to improve the practicability of the models. Without any bells and whistles, our QDMN achieves new state-of-the-art performance on both DAVIS and YouTube-VOS benchmarks. Moreover, extensive experiments demonstrate that the proposed Quality Assessment Module (QAM) can be applied to memory-based methods as generic plugins and significantly improves performance. Our source code is available at https://github.com/workforai/QDMN.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑