噪声设置下的相似性泛化：DIBS现象
Generalizing similarity in noisy setups: the DIBS phenomenon
AI总结：
本文研究SNN在PLN与SLN下的泛化，发现数据密度会在过参数化区域引发DIBS，使PLN导致宏观相似性破坏并阻断完全插值。
摘要：
本工作揭示了相似性学习中数据密度、噪声与泛化能力之间的相互作用。我们考虑孪生神经网络（Siamese Neural Networks，SNN），它是对比学习的基本形式，并探究两类可能影响SNN的噪声：成对标签噪声（Pair Label Noise，PLN）和单标签噪声（Single Label Noise，SLN）。研究表明，无论训练设置如何，SNN都会表现出双下降行为，而且噪声会进一步加剧这一现象。我们证明，数据对的密度对泛化至关重要。当SNN在具有相同PLN或SLN数量的稀疏数据集上训练时，它们表现出相当的泛化特性。然而，在使用密集数据集时，在过参数化区域，PLN情形的泛化比SLN情形更差，从而产生一种我们称为密度诱导的相似性断裂（Density-Induced Break of Similarity，DIBS）的现象。在这一机制下，PLN相似性违背会变成宏观现象，破坏数据集，以至于无论模型参数数量多少，都无法实现完全插值。我们的分析还深入探讨了相似性学习中在线优化与离线泛化之间的对应关系。结果表明，在所考虑的所有场景中，当存在标签噪声时，这种等价性都会失效。
英文摘要：
This work uncovers an interplay among data density, noise, and the generalization ability in similarity learning. We consider Siamese Neural Networks (SNNs), which are the basic form of contrastive learning, and explore two types of noise that can impact SNNs, Pair Label Noise (PLN) and Single Label Noise (SLN). Our investigation reveals that SNNs exhibit double descent behaviour regardless of the training setup and that it is further exacerbated by noise. We demonstrate that the density of data pairs is crucial for generalization. When SNNs are trained on sparse datasets with the same amount of PLN or SLN, they exhibit comparable generalization properties. However, when using dense datasets, PLN cases generalize worse than SLN ones in the overparametrized region, leading to a phenomenon we call Density-Induced Break of Similarity (DIBS). In this regime, PLN similarity violation becomes macroscopical, corrupting the dataset to the point where complete interpolation cannot be achieved, regardless of the number of model parameters. Our analysis also delves into the correspondence between online optimization and offline generalization in similarity learning. The results show that this equivalence fails in the presence of label noise in all the scenarios considered.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
