通过部分感知表示学习理解自监督预训练
Understanding Self-Supervised Pretraining with Part-Aware Representation Learning
AI总结：
本文通过部分感知表示学习视角解释自监督预训练，指出对比学习是部分到整体任务、掩码图像建模是部分到部分任务，并实验证明自监督方法在部分级识别上优于全监督方法，二者结合可进一步提升性能。
摘要：
本文旨在通过研究自监督表示预训练方法学习部分感知表示的能力来理解自监督预训练。该研究的主要动机是：对比学习中使用的随机视图以及掩码图像建模中使用的随机掩码（可见）图像块通常与物体部分有关。我们解释称，对比学习是一个“部分到整体”的任务：投影层根据编码器学习到的物体部分表示来“想象”出整个物体的表示；而掩码图像建模是一个“部分到部分”的任务：物体的掩码图像块由可见图像块“想象”生成。这一解释表明，自监督预训练编码器需要理解物体部分。我们在物体级识别和部分级识别上，对使用若干代表性方法预训练的现成编码器进行了实证比较。结果表明，全监督模型在物体级识别上优于自监督模型，而大多数自监督对比学习和掩码图像建模方法在部分级识别上优于全监督方法。同时观察到，将对比学习与掩码图像建模相结合可以进一步提升性能。
英文摘要：
In this paper, we are interested in understanding self-supervised pretraining through studying the capability that self-supervised representation pretraining methods learn part-aware representations. The study is mainly motivated by that random views, used in contrastive learning, and random masked (visible) patches, used in masked image modeling, are often about object parts. We explain that contrastive learning is a part-to-whole task: the projection layer hallucinates the whole object representation from the object part representation learned from the encoder, and that masked image modeling is a part-to-part task: the masked patches of the object are hallucinated from the visible patches. The explanation suggests that the self-supervised pretrained encoder is required to understand the object part. We empirically compare the off-the-shelf encoders pretrained with several representative methods on object-level recognition and part-level recognition. The results show that the fully-supervised model outperforms self-supervised models for object-level recognition, and most self-supervised contrastive learning and masked image modeling methods outperform the fully-supervised method for part-level recognition. It is observed that the combination of contrastive learning and masked image modeling further improves the performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
