面向零样本语义理解的掩码动量对比学习
Masked Momentum Contrastive Learning for Zero-shot Semantic Understanding
AI总结：
针对大模型微调成本高的问题，提出结合掩码图像建模、动量自蒸馏与全局对比的MMC自监督预训练方法，无需微调即可提升零样本语义分割性能。
摘要：
自监督预训练（SSP）已成为机器学习领域的热门技术，无需标注数据即可提取有意义的特征表示。在计算机视觉领域，预训练视觉Transformer（ViT）在推动迁移学习发展方面发挥了关键作用。然而，由于模型规模的爆炸式增长，微调这些大模型的成本不断攀升，带来了挑战。本研究旨在评估纯自监督学习（SSL）技术在计算机视觉任务中的有效性，无需进行微调，旨在模拟人类对未见过物体的泛化和识别能力。为此，我们提出了一种基于提示补丁的零样本分割评估协议。给定目标物体上的一个点作为提示，该算法计算所选补丁与其他补丁之间的相似度图，在此基础上应用简单的阈值处理来分割目标。另一项评估是物体内和物体间相似度，以衡量SSP ViT的判别能力。从提示式零样本分割和SSP的判别能力中得到的启发，催生了一种名为MMC的简单SSP方法的设计。该方法结合了用于增强局部特征相似性的掩码图像建模、用于将语义从全局特征传递到局部特征的基于动量的自蒸馏，以及用于提升全局特征语义性的全局对比，以增强SSP ViT的判别性表示。因此，我们提出的方法显著降低了物体内和物体间相似度的重叠度，从而实现了图像中有效的物体分割。我们的实验表明，MMC在多个数据集上的零样本语义分割任务中取得了顶级结果。
英文摘要：
Self-supervised pretraining (SSP) has emerged as a popular technique in machine learning, enabling the extraction of meaningful feature representations without labelled data. In the realm of computer vision, pretrained vision transformers (ViTs) have played a pivotal role in advancing transfer learning. Nonetheless, the escalating cost of finetuning these large models has posed a challenge due to the explosion of model size. This study endeavours to evaluate the effectiveness of pure self-supervised learning (SSL) techniques in computer vision tasks, obviating the need for finetuning, with the intention of emulating human-like capabilities in generalisation and recognition of unseen objects. To this end, we propose an evaluation protocol for zero-shot segmentation based on a prompting patch. Given a point on the target object as a prompt, the algorithm calculates the similarity map between the selected patch and other patches, upon that, a simple thresholding is applied to segment the target. Another evaluation is intra-object and inter-object similarity to gauge discriminatory ability of SSP ViTs. Insights from zero-shot segmentation from prompting and discriminatory abilities of SSP led to the design of a simple SSP approach, termed MMC. This approaches combines Masked image modelling for encouraging similarity of local features, Momentum based self-distillation for transferring semantics from global to local features, and global Contrast for promoting semantics of global features, to enhance discriminative representations of SSP ViTs. Consequently, our proposed method significantly reduces the overlap of intra-object and inter-object similarities, thereby facilitating effective object segmentation within an image. Our experiments reveal that MMC delivers top-tier results in zero-shot semantic segmentation across various datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
