鲁棒对比学习的泛化界：从理论到实践
Generalization Bounds for Robust Contrastive Learning: From Theory to Practice
AI总结：
本文为对抗对比学习建立严格泛化理论，揭示良性对比损失与全局散度可提升鲁棒监督损失，并通过实验验证。
摘要：
对比学习首先从无标签数据中提取特征，随后使用有标签数据进行线性探测。对抗对比学习（ACL）将对抗训练整合到第一阶段，以增强特征在探测阶段对抗攻击的鲁棒性。尽管ACL已展现出强大的实证结果，但其理论理解仍然有限。此外，虽然相当多的理论工作分析了无监督损失如何在探测阶段支持监督损失，但尚无研究考察其对鲁棒监督损失的作用。为填补这一空白，我们的工作发展了严格的理论，以识别无监督训练中哪些组件有助于改善鲁棒监督损失。具体而言，除了对抗对比损失外，我们揭示良性对比损失以及良性样本与对抗样本之间的全局散度也能提升鲁棒性。我们进行了适当的实验以验证我们的发现。
英文摘要：
Contrastive Learning first extracts features from unlabeled data, followed by linear probing with labeled data. Adversarial Contrastive Learning (ACL) integrates Adversarial Training into the first phase to enhance feature robustness against attacks in the probing phase. While ACL has shown strong empirical results, its theoretical understanding remains limited. Furthermore, while a fair amount of theoretical works analyze how the unsupervised loss can support the supervised loss in the probing phase, none has examined its role to the robust supervised loss. To fill this gap, our work develops rigorous theories to identify which components in the unsupervised training can help improve the robust supervised loss. Specifically, besides the adversarial contrastive loss, we reveal that the benign one, along with a global divergence between benign and adversarial examples can also improve robustness. Proper experiments are conducted to justify our findings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
