arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.02570cs.LG
收藏

通过动态损失阈值从有噪标签中学习

Learning from Noisy Labels via Dynamic Loss Thresholding

Huawei Technologies(华为技术有限公司);Southeast University(东南大学);Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education(计算机网络和信息集成教育部重点实验室(东南大学));Collaborative Innovation Center of Wireless Communications Technology(无线通信技术协同创新中心)

Hao Yang, Youzhi Jin, Ziyin Li, Deng-Bao Wang, Lei Miao, Xin Geng, Min-Ling Zhang

摘要:

大量的研究已经证明,深度神经网络(DNN)最终可以拟合所有数据,即使是带有噪声标签的数据,但泛化性能很差。然而,最近的研究表明,DNN倾向于逐渐记住数据,从正确的数据转移到错误标记的数据。受此启发,我们提出了一种新的方法,称为动态损失保持(DLT)。在训练过程中,DLT记录每个样本的损失值并计算动态损失阈值。具体地,DLT将每个样本的损失值与当前损失阈值进行比较。具有较小损失的样本可以被认为是具有较高概率的干净样本,反之亦然。然后,DLT丢弃潜在损坏的标签,并进一步利用监督学习技术。在CIFAR-10/100和Clothing 1 M上的实验表明,与最近的最先进的方法相比,有了很大的改进。此外,我们首次研究了两个现实世界的问题。首先,我们提出了一种新的方法来估计数据集的噪声率,基于DNN的早期和后期训练阶段之间的损失差异。其次,我们探讨了硬样本(难以区分)对从噪声标签中学习过程的影响。

英文摘要:

Numerous researches have proved that deep neural networks (DNNs) can fit everything in the end even given data with noisy labels, and result in poor generalization performance. However, recent studies suggest that DNNs tend to gradually memorize the data, moving from correct data to mislabeled data. Inspired by this finding, we propose a novel method named Dynamic Loss Thresholding (DLT). During the training process, DLT records the loss value of each sample and calculates dynamic loss thresholds. Specifically, DLT compares the loss value of each sample with the current loss threshold. Samples with smaller losses can be considered as clean samples with higher probability and vice versa. Then, DLT discards the potentially corrupted labels and further leverages supervised learning techniques. Experiments on CIFAR-10/100 and Clothing1M demonstrate substantial improvements over recent state-of-the-art methods. In addition, we investigate two real-world problems for the first time. Firstly, we propose a novel approach to estimate the noise rates of datasets based on the loss difference between the early and late training stages of DNNs. Secondly, we explore the effect of hard samples (which are difficult to be distinguished) on the process of learning from noisy labels.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑