使深度神经网络对标签噪声具有鲁棒性:一种损失校正方法
Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
摘要:
我们提出了一种理论上有基础的方法来训练深度神经网络,包括递归网络,受到类相关标签噪声的影响。我们提出了两个程序的损失校正是不可知的应用程序域和网络架构。只要我们知道每一个类被破坏成另一个类的概率,它们最多相当于矩阵求逆和乘法。我们进一步展示了如何可以估计这些概率,适应最近的噪声估计技术的多类设置,从而提供了一个端到端的框架。在MNIST,IMDB,CIFAR-10,CIFAR-100和采用多种架构的大规模服装图像数据集上进行了广泛的实验-堆叠密集,卷积,池化,dropout,批量归一化,单词嵌入,LSTM和残差层-证明了我们的建议的噪声鲁棒性。顺便说一句,我们还证明了,当ReLU是唯一的非线性时,损失曲率不受类别相关标签噪声的影响。
英文摘要:
We present a theoretically grounded approach to train deep neural networks, including recurrent networks, subject to class-dependent label noise. We propose two procedures for loss correction that are agnostic to both application domain and network architecture. They simply amount to at most a matrix inversion and multiplication, provided that we know the probability of each class being corrupted into another. We further show how one can estimate these probabilities, adapting a recent technique for noise estimation to the multi-class setting, and thus providing an end-to-end framework. Extensive experiments on MNIST, IMDB, CIFAR-10, CIFAR-100 and a large scale dataset of clothing images employing a diversity of architectures --- stacking dense, convolutional, pooling, dropout, batch normalization, word embedding, LSTM and residual layers --- demonstrate the noise robustness of our proposals. Incidentally, we also prove that, when ReLU is the only non-linearity, the loss curvature is immune to class-dependent label noise.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。