arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.13245cs.CV
收藏

CrOC:面向密集视觉表示学习的跨视图在线聚类

CrOC: Cross-View Online Clustering for Dense Visual Representation Learning

CHUV(洛桑大学医院);EPFL(洛桑联邦理工学院);KU Leuven(鲁汶大学)

Thomas Stegmüller, Tim Lebailly, Behzad Bozorgtabar, Tinne Tuytelaars, Jean-Philippe Thiran

摘要:

学习没有标签的密集视觉表示是一项艰巨的任务,从以场景为中心的数据中学习更是如此。我们建议解决这个具有挑战性的问题,提出了一个跨视图一致性的目标与在线聚类机制(CrOC)发现和分割的语义的意见。在没有手工制作的先验知识的情况下,所得到的方法更具有普遍性,并且不需要繁琐的预处理步骤。更重要的是,聚类算法联合地对两个视图的特征进行操作,从而优雅地绕过了两个视图中未表示的内容以及对象从一个作物到另一个作物的模糊匹配的问题。我们在各种数据集上的线性和无监督分割传输任务以及视频对象分割上表现出出色的性能。我们的代码和预训练模型可在https://github.com/stegmuel/CrOC上公开获取。

英文摘要:

Learning dense visual representations without labels is an arduous task and more so from scene-centric data. We propose to tackle this challenging problem by proposing a Cross-view consistency objective with an Online Clustering mechanism (CrOC) to discover and segment the semantics of the views. In the absence of hand-crafted priors, the resulting method is more generalizable and does not require a cumbersome pre-processing step. More importantly, the clustering algorithm conjointly operates on the features of both views, thereby elegantly bypassing the issue of content not represented in both views and the ambiguous matching of objects from one crop to the other. We demonstrate excellent performance on linear and unsupervised segmentation transfer tasks on various datasets and similarly for video object segmentation. Our code and pre-trained models are publicly available at https://github.com/stegmuel/CrOC.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑