arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2002.06736cs.CV
收藏

基于方向深度嵌入和外观学习的快速视频对象分割

Directional Deep Embedding and Appearance Learning for Fast Video Object Segmentation

Research Center of Precision Sensing and Control, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所精密感知与控制研究中心)

Yingjie Yin, De Xu, Xingang Wang, Lei Zhang

摘要:

最新的半监督视频对象分割(VOS)方法依赖于使用第一帧的给定掩模或后续帧的预测掩模在线微调深度卷积神经网络。然而,在线微调过程通常是耗时的,限制了这种方法的实际使用。我们提出了一种方向性深度嵌入和外观学习(DDEAL)方法,该方法无需在线微调过程,用于快速VOS。首先,提出了一个全局方向匹配模块,该模块可以通过并行卷积操作有效地实现,以学习语义像素嵌入作为内部指导。其次,提出了一种有效的基于统计的方向外观模型来表示VOS的球形嵌入空间上的目标和背景。DDEAL通过全局方向匹配模块和方向外观模型学习模块,从标记的第一帧中学习静态线索,并动态更新后续帧的线索进行对象分割。我们的方法具有最先进的VOS性能,而无需使用在线微调。具体而言,它在DAVIS 2017数据集上获得了74.8%的J & F平均得分,在大规模YouTube-VOS数据集上获得了71.3%的总得分G,同时使用单个NVIDIA TITAN Xp GPU保持了25 fps的速度。此外,我们更快的版本运行31 fps,只有一点精度损失。我们的代码和经过训练的网络可在https://github.com/YingjieYin/Directional-Deep-Embedding-and-Appearance-Learning-for-Fast-Video-Object-Segmentation上获得。

英文摘要:

Most recent semi-supervised video object segmentation (VOS) methods rely on fine-tuning deep convolutional neural networks online using the given mask of the first frame or predicted masks of subsequent frames. However, the online fine-tuning process is usually time-consuming, limiting the practical use of such methods. We propose a directional deep embedding and appearance learning (DDEAL) method, which is free of the online fine-tuning process, for fast VOS. First, a global directional matching module, which can be efficiently implemented by parallel convolutional operations, is proposed to learn a semantic pixel-wise embedding as an internal guidance. Second, an effective directional appearance model based statistics is proposed to represent the target and background on a spherical embedding space for VOS. Equipped with the global directional matching module and the directional appearance model learning module, DDEAL learns static cues from the labeled first frame and dynamically updates cues of the subsequent frames for object segmentation. Our method exhibits state-of-the-art VOS performance without using online fine-tuning. Specifically, it achieves a J & F mean score of 74.8% on DAVIS 2017 dataset and an overall score G of 71.3% on the large-scale YouTube-VOS dataset, while retaining a speed of 25 fps with a single NVIDIA TITAN Xp GPU. Furthermore, our faster version runs 31 fps with only a little accuracy loss. Our code and trained networks are available at https://github.com/YingjieYin/Directional-Deep-Embedding-and-Appearance-Learning-for-Fast-Video-Object-Segmentation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑