MixMask：重新审视Siamese ConvNets的掩蔽策略
MixMask: Revisiting Masking Strategy for Siamese ConvNets
AI总结：
本文提出MixMask，一种基于填充的掩蔽策略，通过替换擦除区域和自适应损失函数，提升Siamese ConvNets在自监督学习中的性能，超越MSCN。
摘要：
自监督学习的最新进展已成功地将掩蔽图像建模（MIM）与Siamese网络相结合，利用了这两种方法的优势。然而，在Siamese ConvNets中集成传统的基于擦除的掩蔽时，仍存在某些挑战。两个主要问题是：（1）ConvNets的连续数据处理特性不允许排除非信息性的掩蔽区域，导致与ViT架构相比训练效率降低；（2）基于擦除的掩蔽与基于对比的目标之间的错位，使其与MIM技术区分开来。为了解决这些挑战，这项工作引入了一种新颖的基于填充的掩蔽方法，称为\textbf{MixMask}。所提出的方法用来自不同图像的内容替换被擦除的区域，有效对抗传统掩蔽方法中看到的信息枯竭。此外，我们揭示了一种自适应损失函数，该函数捕获新修补视图的语义，确保在架构框架内无缝集成。我们通过在不同数据集和应用场景中的全面实验，实证验证了我们方法的有效性。研究结果强调了我们的框架在线性探测、半监督和全监督微调、目标检测和分割等方面的增强性能。值得注意的是，我们的方法超越了MSCN，使MixMask成为Siamese ConvNets更有利的掩蔽解决方案。我们的代码和模型可在https://github.com/kirill-vish/MixMask公开获取。
英文摘要：
The recent progress in self-supervised learning has successfully combined Masked Image Modeling (MIM) with Siamese Networks, harnessing the strengths of both methodologies. Nonetheless, certain challenges persist when integrating conventional erase-based masking within Siamese ConvNets. Two primary concerns are: (1) The continuous data processing nature of ConvNets, which doesn't allow for the exclusion of non-informative masked regions, leading to reduced training efficiency compared to ViT architecture; (2) The misalignment between erase-based masking and the contrastive-based objective, distinguishing it from the MIM technique. To address these challenges, this work introduces a novel filling-based masking approach, termed \textbf{MixMask}. The proposed method replaces erased areas with content from a different image, effectively countering the information depletion seen in traditional masking methods. Additionally, we unveil an adaptive loss function that captures the semantics of the newly patched views, ensuring seamless integration within the architectural framework. We empirically validate the effectiveness of our approach through comprehensive experiments across various datasets and application scenarios. The findings underscore our framework's enhanced performance in areas such as linear probing, semi-supervised and supervised finetuning, object detection and segmentation. Notably, our method surpasses the MSCN, establishing MixMask as a more advantageous masking solution for Siamese ConvNets. Our code and models are publicly available at https://github.com/kirill-vish/MixMask.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
