EnSiam：基于集成表示的自监督学习
EnSiam: Self-Supervised Learning With Ensemble Representations
AI总结：
EnSiam受知识蒸馏中集成方法启发，利用集成表示提供稳定伪标签来改进对比学习，有效缓解了SimSiam对训练配置敏感的问题，并在ImageNet等实验中取得更优性能。
摘要：
最近，对比自监督学习（根据样本身份确定表示的邻近度）在无监督表示学习中取得了显著进展。SimSiam是该领域的著名示例，以其简单却强大的性能著称。然而，由于其结构特性，已知它对训练配置（如超参数和增强设置）的变化较为敏感。为解决此问题，我们关注对比学习与知识蒸馏中教师-学生框架之间的相似性。受基于集成的知识蒸馏方法启发，所提出的方法EnSiam旨在利用集成表示来改进对比学习过程。这能提供稳定的伪标签，从而带来更好的性能。实验表明，EnSiam在大多数情况下优于先前的最先进方法，包括在ImageNet上的实验，这表明EnSiam能够学习高质量表示。
英文摘要：
Recently, contrastive self-supervised learning, where the proximity of representations is determined based on the identities of samples, has made remarkable progress in unsupervised representation learning. SimSiam is a well-known example in this area, known for its simplicity yet powerful performance. However, it is known to be sensitive to changes in training configurations, such as hyperparameters and augmentation settings, due to its structural characteristics. To address this issue, we focus on the similarity between contrastive learning and the teacher-student framework in knowledge distillation. Inspired by the ensemble-based knowledge distillation approach, the proposed method, EnSiam, aims to improve the contrastive learning procedure using ensemble representations. This can provide stable pseudo labels, providing better performance. Experiments demonstrate that EnSiam outperforms previous state-of-the-art methods in most cases, including the experiments on ImageNet, which shows that EnSiam is capable of learning high-quality representations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
