面向高级掩码自编码器学习的协同掩码与目标动态双雄
The Dynamic Duo of Collaborative Masking and Target for Advanced Masked Autoencoder Learning
AI总结：
本文提出CMT-MAE框架，通过教师与学生模型注意力线性聚合的协同掩码机制及双模型输出特征作为解码器协同目标，在ImageNet-1K上预训练显著提升了MAE的微调与线性探测性能。
摘要：
掩码自编码器（MAE）近期在自监督视觉表征学习中取得了成功。以往工作主要采用定制化（如随机、分块）掩码或教师（如CLIP）引导的掩码和目标。然而，它们忽略了自训练（学生）模型在向教师提供掩码和目标反馈方面的潜在作用。在本工作中，我们提出整合协同掩码与目标以增强掩码自编码器，即CMT-MAE。具体而言，CMT-MAE通过对教师和学生模型的注意力进行线性聚合，利用了一种简单的协同掩码机制。我们进一步提出将这两个模型的输出特征作为解码器的协同目标。我们在ImageNet-1K上预训练的简单有效框架取得了最先进的线性探测和微调性能。特别是，使用ViT-base，我们将原始MAE的微调结果从83.6%提升至85.7%。
英文摘要：
Masked autoencoders (MAE) have recently succeeded in self-supervised vision representation learning. Previous work mainly applied custom-designed (e.g., random, block-wise) masking or teacher (e.g., CLIP)-guided masking and targets. However, they ignore the potential role of the self-training (student) model in giving feedback to the teacher for masking and targets. In this work, we present to integrate Collaborative Masking and Targets for boosting Masked AutoEncoders, namely CMT-MAE. Specifically, CMT-MAE leverages a simple collaborative masking mechanism through linear aggregation across attentions from both teacher and student models. We further propose using the output features from those two models as the collaborative target of the decoder. Our simple and effective framework pre-trained on ImageNet-1K achieves state-of-the-art linear probing and fine-tuning performance. In particular, using ViT-base, we improve the fine-tuning results of the vanilla MAE from 83.6% to 85.7%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
