arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.09059cs.CV
收藏

耦合全局上下文与局部内容用于弱监督语义分割

Coupling Global Context and Local Contents for Weakly-Supervised Semantic Segmentation

Nanjing University of Science and Technology(南京理工大学);Nanjing University of Aeronautics and Astronautics(南京航空航天大学)

Chunyan Wang, Dong Zhang, Liyan Zhang, Jinhui Tang

AI总结:

本文针对单阶段弱监督语义分割中的背景与目标不完整问题,提出仅用图像级标签监督的 WS-FCN,通过全局上下文聚合与语义一致的局部特征融合实现端到端训练,并在 PASCAL VOC 2012 和 MS COCO 2014 上取得最优 mIoU。

摘要:

得益于友好标注和令人满意的性能优势,弱监督语义分割(WSSS)方法已得到广泛研究。近来,单阶段 WSSS 被提出以缓解多阶段 WSSS 中计算成本高昂和训练流程复杂的问题。然而,这种尚不成熟的模型所得结果存在背景不完整和目标不完整的问题。我们通过实验发现,它们分别由全局目标上下文不足以及局部区域内容缺乏造成。基于这些观察,我们提出一种仅使用图像级类别标签监督的单阶段 WSSS 模型,称为 Weakly Supervised Feature Coupling Network(WS-FCN,弱监督特征耦合网络),它能够捕获由相邻特征网格形成的多尺度上下文,并将来自低层特征的细粒度空间信息编码到高层特征中。具体而言,我们提出一个灵活的上下文聚合模块,以在不同粒度空间中捕获全局目标上下文。此外,我们以自底向上、参数可学习的方式提出一个语义一致的特征融合模块,用于聚合细粒度局部内容。基于这两个模块,WS-FCN 采用自监督的端到端训练方式。在具有挑战性的 PASCAL VOC 2012 和 MS COCO 2014 上的大量实验结果证明了 WS-FCN 的有效性和效率:该模型在 PASCAL VOC 2012 验证集和测试集上分别取得 65.02% 和 64.22% 的 mIoU,在 MS COCO 2014 验证集上取得 34.12% 的 mIoU,达到当前最优结果。代码和权重已发布于:https://github.com/ChunyanWang1/ws-fcn。

英文摘要:

Thanks to the advantages of the friendly annotations and the satisfactory performance, Weakly-Supervised Semantic Segmentation (WSSS) approaches have been extensively studied. Recently, the single-stage WSSS was awakened to alleviate problems of the expensive computational costs and the complicated training procedures in multi-stage WSSS. However, results of such an immature model suffer from problems of background incompleteness and object incompleteness. We empirically find that they are caused by the insufficiency of the global object context and the lack of the local regional contents, respectively. Under these observations, we propose a single-stage WSSS model with only the image-level class label supervisions, termed as Weakly Supervised Feature Coupling Network (WS-FCN), which can capture the multi-scale context formed from the adjacent feature grids, and encode the fine-grained spatial information from the low-level features into the high-level ones. Specifically, a flexible context aggregation module is proposed to capture the global object context in different granular spaces. Besides, a semantically consistent feature fusion module is proposed in a bottom-up parameter-learnable fashion to aggregate the fine-grained local contents. Based on these two modules, WS-FCN lies in a self-supervised end-to-end training fashion. Extensive experimental results on the challenging PASCAL VOC 2012 and MS COCO 2014 demonstrate the effectiveness and efficiency of WS-FCN, which can achieve state-of-the-art results by 65.02\% and 64.22\% mIoU on PASCAL VOC 2012 val set and test set, 34.12\% mIoU on MS COCO 2014 val set, respectively. The code and weight have been released at:https://github.com/ChunyanWang1/ws-fcn.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑