arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2112.02853cs.CV
收藏

用于视频对象分割的可靠消隐-校正调制

Reliable Propagation-Correction Modulation for Video Object Segmentation

Huangzhong University of Science & Technology(华中科技大学);Microsoft Research Asia(微软亚洲研究院)

Xiaohao Xu, Jinglu Wang, Xiao Li, Yan Lu

摘要:

误差传播是在线半监督视频对象分割中的一个普遍而关键的问题。我们的目标是通过具有高可靠性的校正机制来抑制错误传播。关键的洞察力是解开传统的掩模传播过程中的可靠线索的校正。我们引入了两个调制器,传播和校正调制器,分别根据本地时间相关性和可靠的参考对目标帧嵌入进行通道级重新校准。具体来说,我们组装的调制器与级联的传播校正方案。这避免了传播调制器对可靠校正调制器的影响的覆盖。虽然具有地面实况标签的参考帧提供了可靠的线索,但它可能与目标帧非常不同,并引入不确定或不完整的相关性。我们通过补充可靠的特征补丁来增强参考线索,从而为调制器提供更全面和更具表现力的对象表示。此外,一个可靠性过滤器的设计,以检索可靠的补丁,并通过他们在随后的帧。我们的模型在YouTube-VOS18/19和DAVIS17-Val/Test基准测试中达到了最先进的性能。大量的实验表明,校正机制提供了相当大的性能增益,充分利用可靠的指导。代码可在:www.example.com.

英文摘要:

Error propagation is a general but crucial problem in online semi-supervised video object segmentation. We aim to suppress error propagation through a correction mechanism with high reliability. The key insight is to disentangle the correction from the conventional mask propagation process with reliable cues. We introduce two modulators, propagation and correction modulators, to separately perform channel-wise re-calibration on the target frame embeddings according to local temporal correlations and reliable references respectively. Specifically, we assemble the modulators with a cascaded propagation-correction scheme. This avoids overriding the effects of the reliable correction modulator by the propagation modulator. Although the reference frame with the ground truth label provides reliable cues, it could be very different from the target frame and introduce uncertain or incomplete correlations. We augment the reference cues by supplementing reliable feature patches to a maintained pool, thus offering more comprehensive and expressive object representations to the modulators. In addition, a reliability filter is designed to retrieve reliable patches and pass them in subsequent frames. Our model achieves state-of-the-art performance on YouTube-VOS18/19 and DAVIS17-Val/Test benchmarks. Extensive experiments demonstrate that the correction mechanism provides considerable performance gain by fully utilizing reliable guidance. Code is available at: https://github.com/JerryX1110/RPCMVOS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑