arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.01075cs.CV
收藏

基于边界知识翻译的参考语义分割

Boundary Knowledge Translation based Reference Semantic Segmentation

Zhejiang Lab(之江实验室);Zhejiang University(浙江大学);National University of Singapore(新加坡国立大学);Zhejiang University Of Technology(浙江工业大学)

Lechao Cheng, Zunlei Feng, Xinchao Wang, Ya Jie Liu, Jie Lei, Mingli Song

摘要:

给定图像中未知类型的参考对象,人类观察者可以毫不费力地在另一幅图像中找到相同类别的对象,并精确地说出它们的视觉边界。人类的这种视觉认知能力似乎在当前计算机视觉的研究范围中是不存在的。例如,现有的分割网络依赖于大量的标记数据,收集和注释这些数据既费力又昂贵;此外,随着类别数量的增加,分割网络的性能往往会下降。在本文中,我们引入了一种新的参考语义分割网络(Ref-Net)进行视觉边界知识翻译。Ref-Net包含一个参考分割模块(RSM)和一个边界知识翻译模块(BKTM)。受人类识别机制的启发,RSM仅根据参考对象的特征来分割相同类别的对象。BKTM则引入两个边界映射分支,以对抗的方式对目标对象进行内外边界分割,并将开源数据集的标注边界知识转化为分割网络。详尽的实验表明,在数十个细粒度注释样本的指导下,Ref-Net在六个数据集上实现了与完全监督方法相当的结果。

英文摘要:

Given a reference object of an unknown type in an image, human observers can effortlessly find the objects of the same category in another image and precisely tell their visual boundaries. Such visual cognition capability of humans seems absent from the current research spectrum of computer vision. Existing segmentation networks, for example, rely on a humongous amount of labeled data, which is laborious and costly to collect and annotate; besides, the performance of segmentation networks tend to downgrade as the number of the category increases. In this paper, we introduce a novel Reference semantic segmentation Network (Ref-Net) to conduct visual boundary knowledge translation. Ref-Net contains a Reference Segmentation Module (RSM) and a Boundary Knowledge Translation Module (BKTM). Inspired by the human recognition mechanism, RSM is devised only to segment the same category objects based on the features of the reference objects. BKTM, on the other hand, introduces two boundary discriminator branches to conduct inner and outer boundary segmentation of the target objectin an adversarial manner, and translate the annotated boundary knowledge of open-source datasets into the segmentation network. Exhaustive experiments demonstrate that, with tens of finely-grained annotated samples as guidance, Ref-Net achieves results on par with fully supervised methods on six datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑