All4One：通过自注意力和冗余削减的共生邻居对比学习
All4One: Symbiotic Neighbour Contrastive Learning via Self-Attention and Redundancy Reduction
AI总结：
本文提出All4One，一种基于质心自注意力和多目标对比的自监督学习方法，在多个基准上超越实例判别方法，并在低维弱增强下优于NNCLR和Barlow Twins。
摘要：
基于最近邻的方法已被证明是最成功的自监督学习（SSL）方法之一，这得益于其高泛化能力。然而，当使用多个邻居时，其计算效率会降低。在本文中，我们提出了一种新颖的对比自监督学习方法，称为All4One，该方法通过自注意力机制创建的“质心”来减少邻居表示之间的距离。我们使用质心对比目标以及单一邻居对比和特征对比目标。质心有助于从多个邻居中学习上下文信息，而邻居对比则直接从邻居学习表示，特征对比则学习特征特有的表示。这种组合使All4One在流行基准数据集的线性分类评估中优于流行的实例判别方法超过1%，并取得了最先进（SoTA）的结果。最后，我们表明All4One对嵌入维度和增强具有鲁棒性，在低维度和弱增强设置下，其性能超过NNCLR和Barlow Twins超过5%。源代码即将公开。
英文摘要：
Nearest neighbour based methods have proved to be one of the most successful self-supervised learning (SSL) approaches due to their high generalization capabilities. However, their computational efficiency decreases when more than one neighbour is used. In this paper, we propose a novel contrastive SSL approach, which we call All4One, that reduces the distance between neighbour representations using ''centroids'' created through a self-attention mechanism. We use a Centroid Contrasting objective along with single Neighbour Contrasting and Feature Contrasting objectives. Centroids help in learning contextual information from multiple neighbours whereas the neighbour contrast enables learning representations directly from the neighbours and the feature contrast allows learning representations unique to the features. This combination enables All4One to outperform popular instance discrimination approaches by more than 1% on linear classification evaluation for popular benchmark datasets and obtains state-of-the-art (SoTA) results. Finally, we show that All4One is robust towards embedding dimensionalities and augmentations, surpassing NNCLR and Barlow Twins by more than 5% on low dimensionality and weak augmentation settings. The source code would be made available soon.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
