arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.13101cs.CV
收藏

语义分割中对象部分的自监督学习

Self-Supervised Learning of Object Parts for Semantic Segmentation

Technical University of Munich(慕尼黑工业大学);University of Amsterdam(阿姆斯特丹大学)

Adrian Ziegler, Yuki M. Asano

摘要:

自监督学习的进展带来了强大的通用图像表示学习方法。然而,到目前为止,它主要集中在图像级别的学习。反过来,像无监督图像分割这样的任务并没有从这种趋势中受益,因为它们需要空间多样的表示。然而,学习密集表示是具有挑战性的,因为在无监督的上下文中,不清楚如何引导模型学习与各种潜在对象类别相对应的表示。在本文中,我们认为,自监督学习的对象部分是一个解决这个问题。对象部分是可泛化的:它们先验地独立于对象定义,但可以被分组以形成后验的对象。为此,我们利用最近提出的Vision Transformer的出席对象的能力,并将其与空间密集的聚类任务进行微调的空间令牌。我们的方法超过了国家的最先进的三个语义分割基准的17%-3%,表明我们的表示是通用的各种对象定义。最后,我们将其扩展到完全无监督的分割-即使在测试时也完全避免使用标签信息-并证明了一种基于社区发现的自动合并发现对象部分的简单方法会产生很大的收益。

英文摘要:

Progress in self-supervised learning has brought strong general image representation learning methods. Yet so far, it has mostly focused on image-level learning. In turn, tasks such as unsupervised image segmentation have not benefited from this trend as they require spatially-diverse representations. However, learning dense representations is challenging, as in the unsupervised context it is not clear how to guide the model to learn representations that correspond to various potential object categories. In this paper, we argue that self-supervised learning of object parts is a solution to this issue. Object parts are generalizable: they are a priori independent of an object definition, but can be grouped to form objects a posteriori. To this end, we leverage the recently proposed Vision Transformer's capability of attending to objects and combine it with a spatially dense clustering task for fine-tuning the spatial tokens. Our method surpasses the state-of-the-art on three semantic segmentation benchmarks by 17%-3%, showing that our representations are versatile under various object definitions. Finally, we extend this to fully unsupervised segmentation - which refrains completely from using label information even at test-time - and demonstrate that a simple method for automatically merging discovered object parts based on community detection yields substantial gains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑