arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.08421cs.CV
收藏

WegFormer:弱监督语义分割的Transformers

WegFormer: Transformers for Weakly Supervised Semantic Segmentation

Tongji University(同济大学);The University of Hong Kong(香港大学);SenseTime(商汤科技);Shanghai AI Lab(上海人工智能实验室)

Chunmeng Liu, Enze Xie, Wenjia Wang, Wenhai Wang, Guangyao Li, Ping Luo

摘要:

尽管卷积神经网络(CNN)在弱监督语义分割(WSSS)方面取得了显着进展,但CNN的有效感受野不足以捕获全局上下文信息,从而导致次优结果。受Transformers在基础视觉领域的巨大成功的启发,本文首次引入了Transformer来构建一个简单有效的WSSS框架,称为WegFormer。与现有的基于CNN的方法不同,WegFormer使用视觉Transformer(ViT)作为分类器来产生高质量的伪分割掩模。为此,我们在基于transformer的框架中引入了三个定制组件,它们是(1)深度泰勒分解(DTD)以生成注意力地图,(2)软擦除模块以平滑注意力地图,以及(3)高效的潜在对象挖掘(EPOM)以过滤背景中的噪声激活。在没有任何额外功能的情况下,WegFormer在PASCAL VOC数据集上实现了最先进的70.5% mIoU,显著优于之前的最佳方法。我们希望WegFormer为挖掘Transformer在弱监督语义分割中的潜力提供一个新的视角。代码将被释放。

英文摘要:

Although convolutional neural networks (CNNs) have achieved remarkable progress in weakly supervised semantic segmentation (WSSS), the effective receptive field of CNN is insufficient to capture global context information, leading to sub-optimal results. Inspired by the great success of Transformers in fundamental vision areas, this work for the first time introduces Transformer to build a simple and effective WSSS framework, termed WegFormer. Unlike existing CNN-based methods, WegFormer uses Vision Transformer (ViT) as a classifier to produce high-quality pseudo segmentation masks. To this end, we introduce three tailored components in our Transformer-based framework, which are (1) a Deep Taylor Decomposition (DTD) to generate attention maps, (2) a soft erasing module to smooth the attention maps, and (3) an efficient potential object mining (EPOM) to filter noisy activation in the background. Without any bells and whistles, WegFormer achieves state-of-the-art 70.5% mIoU on the PASCAL VOC dataset, significantly outperforming the previous best method. We hope WegFormer provides a new perspective to tap the potential of Transformer in weakly supervised semantic segmentation. Code will be released.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑