arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.14343cs.CVeess.IV
收藏

仅靠记忆匹配是不够的:联合改进记忆匹配与解码以实现视频目标分割

Memory Matching is not Enough: Jointly Improving Memory Matching and Decoding for Video Object Segmentation

University of Chinese Academy of Sciences(中国科学院大学);South China Agricultural University(华南农业大学);Beijing University Of Technology(北京工业大学);Monash University(莫纳什大学)

Jintu Zheng, Yun Liang, Yuqing Zhang, Wanchao Su

AI总结:

针对视频目标分割中记忆匹配易出错和丢失关键信息的问题,提出联合改进匹配与解码阶段的方法,通过代价感知机制、多尺度匹配和解码补偿机制,在多个基准上取得卓越性能。

摘要:

基于记忆的视频目标分割方法通过建立记忆库来对长时空跨度上的多个目标进行建模,取得了显著性能。然而,这些方法难以克服错误匹配问题,容易丢失关键信息,导致不同目标之间产生混淆。本文提出一种联合改进匹配与解码阶段的有效方法,以缓解记忆匹配阶段的错误匹配问题。具体而言,我们提出了一种代价感知机制来抑制短期记忆中的轻微误差,并采用了一种分流的多尺度匹配方法用于长期记忆,以在各种目标尺度下覆盖宽泛的视觉空间。对于解码阶段,我们实现了一种补偿机制,旨在恢复匹配阶段丢失的关键信息。所提方法在多个流行基准上取得了卓越性能:DAVIS 2016&2017 Val(92.4%&88.1%),DAVIS 2017 Test(83.4%),以及在YouTubeVOS 2018&2019 Val上达到84.4%&84.6%。

英文摘要:

Memory-based video object segmentation methods model multiple objects over long temporal-spatial spans by establishing memory bank, which achieve the remarkable performance. However, they struggle to overcome the false matching and are prone to lose critical information, resulting in confusion among different objects. In this paper, we propose an effective approach which jointly improving the matching and decoding stages to alleviate the false matching issue.For the memory matching stage, we present a cost aware mechanism that suppresses the slight errors for short-term memory and a shunted cross-scale matching for long-term memory which establish a wide filed matching spaces for various object scales. For the readout decoding stage, we implement a compensatory mechanism aims at recovering the essential information where missing at the matching stage. Our approach achieves the outstanding performance in several popular benchmarks (i.e., DAVIS 2016&2017 Val (92.4%&88.1%), and DAVIS 2017 Test (83.9%)), and achieves 84.8%&84.6% on YouTubeVOS 2018&2019 Val.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑