arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.05528cs.CV
收藏

准确检测并非应对含噪网络数据集中标签噪声的全部要素

An accurate detection is not all you need to combat label noise in web-noisy datasets

Australian Institute for Machine Learning, University of Adelaide(阿德莱德大学澳大利亚机器学习研究所);CeADAR: Ireland’s Centre for Applied Artificial Intelligence(CeADAR:爱尔兰应用人工智能中心);Insight Centre for Data Analytics, Dublin City University(都柏林城市大学洞察数据分析中心)

Paul Albert, Jack Valmadre, Eric Arazo, Tarun Krishna, Noel E. O'Connor, Kevin McGuinness

AI总结:

针对网络爬取数据的标签噪声问题,研究发现无监督对比学习下的线性OOD检测无法提升分类精度,提出结合线性分离检测与小损失方法的混合方案,显著提升了真实含噪网络图像分类的SOTA结果。

摘要:

在网络爬取数据上训练分类器,需要对标注错误和无关样本具有鲁棒性的学习算法。本文基于近期的经验观察展开研究:对含噪的网络爬取数据集应用无监督对比学习,可得到一种特征表示,在该表示下分布内(ID)样本与分布外(OOD)样本呈线性可分状态。我们证明,直接估计分离超平面确实能够准确检测OOD样本,但令人意外的是,这种检测能力并未转化为分类准确率的提升。深入探究这一现象后,我们发现,近乎完美的检测遗漏了一类对监督学习有价值的干净样本。这类样本通常是视觉上简单的图像,尽管在无监督学习下它们与OOD分布的分离度很差,但使用标准的基于损失或距离的方法,相对容易将其识别为干净样本。由于我们进一步观察到该方法与SOTA(当前最优)指标的相关性较低,因此我们提出了一种混合解决方案:交替使用基于线性分离的噪声检测和SOTA小损失方法。与SOTA算法PLS结合后,我们显著提升了存在网络噪声的真实世界图像分类任务的SOTA结果,相关代码见github.com/PaulAlbert31/LSA。

英文摘要:

Training a classifier on web-crawled data demands learning algorithms that are robust to annotation errors and irrelevant examples. This paper builds upon the recent empirical observation that applying unsupervised contrastive learning to noisy, web-crawled datasets yields a feature representation under which the in-distribution (ID) and out-of-distribution (OOD) samples are linearly separable. We show that direct estimation of the separating hyperplane can indeed offer an accurate detection of OOD samples, and yet, surprisingly, this detection does not translate into gains in classification accuracy. Digging deeper into this phenomenon, we discover that the near-perfect detection misses a type of clean examples that are valuable for supervised learning. These examples often represent visually simple images, which are relatively easy to identify as clean examples using standard loss- or distance-based methods despite being poorly separated from the OOD distribution using unsupervised learning. Because we further observe a low correlation with SOTA metrics, this urges us to propose a hybrid solution that alternates between noise detection using linear separation and a state-of-the-art (SOTA) small-loss approach. When combined with the SOTA algorithm PLS, we substantially improve SOTA results for real-world image classification in the presence of web noise github.com/PaulAlbert31/LSA

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑