arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.04578cs.CVcs.LG
收藏

你的噪音修正器噪音大吗?PLS:两阶段检测对标签噪声的鲁棒性

Is your noise correction noisy? PLS: Robustness to label noise with two stage detection

Dublin City University(都柏林城市大学);Insight SFI Centre for Data Analytics(Insight SFI数据分析中心)

Paul Albert, Eric Arazo, Tarun Krishna, Noel E. O'Connor, Kevin McGuinness

摘要:

设计能够在来自网络的未经策划的数据集上训练准确的神经网络的鲁棒算法一直是许多研究的主题,因为它减少了对耗时的人力劳动的需求。许多以前的研究成果的重点一直是对不同类型的标签噪声的检测,然而,本文提出了提高校正精度的噪声样本,一旦他们被检测到。在许多国家的最先进的贡献,采用两个阶段的方法,其中检测的噪声样本之前猜测一个正确的伪标签在半监督的方式。然后在监督目标中使用猜测的伪标签,而不确保标签猜测可能是正确的。这可能导致确认偏差,从而降低噪声鲁棒性。在这里,我们提出了伪损失,一个简单的度量,我们发现是强相关的伪标签的正确性在嘈杂的样本。使用伪损失,我们在整个训练过程中动态降低置信度不足的伪标签的权重,以避免确认偏差并提高网络准确性。我们还建议使用置信度引导的对比目标,该目标在置信度校正样本的类界(监督)和置信度不足的标签校正的无监督表示之间的插值目标上学习鲁棒表示。实验证明了我们的伪损失选择(PLS)算法在各种基准数据集上的最新性能,包括被分布内和分布外噪声综合破坏的策划数据,以及两个真实世界的网络噪声数据集。我们的实验是完全可重复的github.com/PaulAlbert31/SNCF

英文摘要:

Designing robust algorithms capable of training accurate neural networks on uncurated datasets from the web has been the subject of much research as it reduces the need for time consuming human labor. The focus of many previous research contributions has been on the detection of different types of label noise; however, this paper proposes to improve the correction accuracy of noisy samples once they have been detected. In many state-of-the-art contributions, a two phase approach is adopted where the noisy samples are detected before guessing a corrected pseudo-label in a semi-supervised fashion. The guessed pseudo-labels are then used in the supervised objective without ensuring that the label guess is likely to be correct. This can lead to confirmation bias, which reduces the noise robustness. Here we propose the pseudo-loss, a simple metric that we find to be strongly correlated with pseudo-label correctness on noisy samples. Using the pseudo-loss, we dynamically down weight under-confident pseudo-labels throughout training to avoid confirmation bias and improve the network accuracy. We additionally propose to use a confidence guided contrastive objective that learns robust representation on an interpolated objective between class bound (supervised) for confidently corrected samples and unsupervised representation for under-confident label corrections. Experiments demonstrate the state-of-the-art performance of our Pseudo-Loss Selection (PLS) algorithm on a variety of benchmark datasets including curated data synthetically corrupted with in-distribution and out-of-distribution noise, and two real world web noise datasets. Our experiments are fully reproducible github.com/PaulAlbert31/SNCF

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑