ArCL：通过增强鲁棒表征增强对比学习
ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations
AI总结：
该研究针对自监督对比学习可迁移性理论理解不足的问题，构建理论框架分析数据增强的影响，发现对比学习难学域不变特征，据此提出ArCL方法，可提升对比学习的可迁移性。
摘要：
自监督学习（Self-Supervised Learning, SSL）是一种利用无标注数据训练模型的范式。实证研究表明，在下游任务与训练数据分布存在差异的分布偏移场景下，SSL 能取得可观的性能。然而，学界对其可迁移性的理论理解仍较为有限。本文构建了一套理论框架，通过探究数据增强对自监督对比学习的影响，分析其可迁移性。研究结果显示，对比学习的下游性能很大程度上取决于数据增强的选择。此外我们发现，对比学习无法学习到域不变特征，这限制了它的可迁移性。基于这些理论洞见，我们提出了名为增强鲁棒对比学习（Augmentation-robust Contrastive Learning, ArCL）的新方法，该方法能保证学习到域不变特征，且可轻松集成到现有对比学习算法中。我们在多个数据集上开展了实验，结果表明 ArCL 显著提升了对比学习的可迁移性。
英文摘要：
Self-Supervised Learning (SSL) is a paradigm that leverages unlabeled data for model training. Empirical studies show that SSL can achieve promising performance in distribution shift scenarios, where the downstream and training distributions differ. However, the theoretical understanding of its transferability remains limited. In this paper, we develop a theoretical framework to analyze the transferability of self-supervised contrastive learning, by investigating the impact of data augmentation on it. Our results reveal that the downstream performance of contrastive learning depends largely on the choice of data augmentation. Moreover, we show that contrastive learning fails to learn domain-invariant features, which limits its transferability. Based on these theoretical insights, we propose a novel method called Augmentation-robust Contrastive Learning (ArCL), which guarantees to learn domain-invariant features and can be easily integrated with existing contrastive learning algorithms. We conduct experiments on several datasets and show that ArCL significantly improves the transferability of contrastive learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
