从局部对比视角理解掩码自编码器
Understanding Masked Autoencoders From a Local Contrastive Perspective
AI总结：
本文提出 LC-MAE 框架，从 patch 级局部对比视角解释 MAE 的重建目标，揭示其掩码不变性、分布一致性及解码器与随机掩码的作用。
摘要：
Masked AutoEncoder（MAE，掩码自编码器）以其简单而有效的掩码与重建策略革新了自监督学习领域。然而，尽管 MAE 在各类下游视觉任务中取得了最先进性能，与经典对比学习范式相比，驱动其有效性的底层机制仍未得到充分研究。本文中，我们首先提出一种局部视角，在 patch 层面从 MAE 的重建目标中显式提取出局部对比形式。随后，我们引入一个名为 Local Contrastive MAE（LC-MAE）的新实证框架，以分析 MAE 的重建与对比两个方面。LC-MAE 表明，MAE 学习对随机掩码的不变性，并确保所学 token 嵌入与原始图像之间的分布一致性。此外，我们剖析了解码器和随机掩码对 MAE 成功的贡献，揭示了解码器的学习机制，以及随机掩码作为数据增强和有效感受野限制的双重作用。我们的实验分析阐明了 MAE 的复杂之处，并总结了一些有用的设计方法，这些方法可启发更强大的视觉自监督方法。
英文摘要：
Masked AutoEncoder (MAE) has revolutionized the field of self-supervised learning with its simple yet effective masking and reconstruction strategies. However, despite achieving state-of-the-art performance across various downstream vision tasks, the underlying mechanisms that drive MAE's efficacy are less well-explored compared to the canonical contrastive learning paradigm. In this paper, we first propose a local perspective to explicitly extract a local contrastive form from MAE's reconstructive objective at the patch level. And then we introduce a new empirical framework, called Local Contrastive MAE (LC-MAE), to analyze both reconstructive and contrastive aspects of MAE. LC-MAE reveals that MAE learns invariance to random masking and ensures distribution consistency between the learned token embeddings and the original images. Furthermore, we dissect the contribution of the decoder and random masking to MAE's success, revealing both the decoder's learning mechanism and the dual role of random masking as data augmentation and effective receptive field restriction. Our experimental analysis sheds light on the intricacies of MAE and summarizes some useful design methodologies, which can inspire more powerful visual self-supervised methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
