MIM-Refiner：来自中间预训练表示的对比学习增强
MIM-Refiner: A Contrastive Learning Boost from Intermediate Pre-Trained Representations
AI总结：
本研究提出MIM-Refiner方法，利用连接MIM模型中间层的多对比头与改进最近邻目标构建语义簇，仅需少量训练轮次即可提升预训练MIM模型的下游任务性能，在多个基准上超越现有最先进SSL模型。
摘要：
我们提出MIM（掩码图像建模）-Refiner，一种针对预训练MIM模型的对比学习增强方法。MIM-Refiner的提出源于这样一种洞见：MIM模型中的强表示通常存在于中间层。据此，MIM-Refiner利用连接到不同中间层的多个对比头。在每个对比头中，一种改进的最近邻目标构建语义簇，这些语义簇捕获的语义信息可提升下游任务的性能，包括即插即用（off-the-shelf）和微调设置。\n 该精炼过程简短且简单，但效果极佳。仅需几个epoch，我们就能将MIM模型的特征从次优水平精炼为最先进的即插即用特征。对在ImageNet-1K上用data2vec 2.0预训练的ViT-H进行精炼后，在仅使用ImageNet-1K预训练的模型中，其线性探测（84.7%）和少样本分类性能创下新的最先进水平。MIM-Refiner高效结合了MIM和ID目标的优势，在少样本分类、长尾分类、聚类和语义分割等多种基准上优于此前最先进的自监督学习（SSL）模型。
英文摘要：
We introduce MIM (Masked Image Modeling)-Refiner, a contrastive learning boost for pre-trained MIM models. MIM-Refiner is motivated by the insight that strong representations within MIM models generally reside in intermediate layers. Accordingly, MIM-Refiner leverages multiple contrastive heads that are connected to different intermediate layers. In each head, a modified nearest neighbor objective constructs semantic clusters that capture semantic information which improves performance on downstream tasks, including off-the-shelf and fine-tuning settings. The refinement process is short and simple - yet highly effective. Within a few epochs, we refine the features of MIM models from subpar to state-of-the-art, off-the-shelf features. Refining a ViT-H, pre-trained with data2vec 2.0 on ImageNet-1K, sets a new state-of-the-art in linear probing (84.7%) and low-shot classification among models that are pre-trained on ImageNet-1K. MIM-Refiner efficiently combines the advantages of MIM and ID objectives and compares favorably against previous state-of-the-art SSL models on a variety of benchmarks such as low-shot classification, long-tailed classification, clustering and semantic segmentation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
