arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1609.03683stat.MLcs.LG
收藏

使深度神经网络对标签噪声具有鲁棒性:一种损失校正方法

Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach

Australian National University(澳大利亚国立大学);Data61;Waynaut;University of Sydney(悉尼大学)

Giorgio Patrini, Alessandro Rozza, Aditya Menon, Richard Nock, Lizhen Qu

摘要:

我们提出了一种理论上有基础的方法来训练深度神经网络,包括递归网络,受到类相关标签噪声的影响。我们提出了两个程序的损失校正是不可知的应用程序域和网络架构。只要我们知道每一个类被破坏成另一个类的概率,它们最多相当于矩阵求逆和乘法。我们进一步展示了如何可以估计这些概率,适应最近的噪声估计技术的多类设置,从而提供了一个端到端的框架。在MNIST,IMDB,CIFAR-10,CIFAR-100和采用多种架构的大规模服装图像数据集上进行了广泛的实验-堆叠密集,卷积,池化,dropout,批量归一化,单词嵌入,LSTM和残差层-证明了我们的建议的噪声鲁棒性。顺便说一句,我们还证明了,当ReLU是唯一的非线性时,损失曲率不受类别相关标签噪声的影响。

英文摘要:

We present a theoretically grounded approach to train deep neural networks, including recurrent networks, subject to class-dependent label noise. We propose two procedures for loss correction that are agnostic to both application domain and network architecture. They simply amount to at most a matrix inversion and multiplication, provided that we know the probability of each class being corrupted into another. We further show how one can estimate these probabilities, adapting a recent technique for noise estimation to the multi-class setting, and thus providing an end-to-end framework. Extensive experiments on MNIST, IMDB, CIFAR-10, CIFAR-100 and a large scale dataset of clothing images employing a diversity of architectures --- stacking dense, convolutional, pooling, dropout, batch normalization, word embedding, LSTM and residual layers --- demonstrate the noise robustness of our proposals. Incidentally, we also prove that, when ReLU is the only non-linearity, the loss curvature is immune to class-dependent label noise.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑