arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.05046cs.LG
收藏

一种面向含噪声标签的在线鲁棒深度神经网络训练的基于梯度的方法

A Gradient-based Approach for Online Robust Deep Neural Network Training with Noisy Labels

University of California(加利福尼亚大学);JP Morgan Chase AI Research(摩根大通人工智能研究院)

Yifan Yang, Alec Koppel, Zheng Zhang

AI总结:

针对在线流数据场景下的噪声标签学习问题,提出基于梯度的OGRS方法,可自动筛选干净样本增量更新模型,经理论证明收敛且实验效果优于现有最优方法。

摘要:

在许多真实场景中,带噪声标签的学习是可扩展训练的重要研究课题,但以往研究很少关注数据以流形式到达的在线场景下的该问题。本文提出了一种新颖的基于梯度的方法,用于模型参数在线学习过程中的噪声标签检测,名为Online Gradient-based Robust Selection(在线梯度鲁棒选择,简称OGRS)。与以往离线训练的样本选择方法需要在每轮训练前估算数据集的干净样本比例不同,OGRS无需调整参数设置,就能通过梯度更新步骤从干净比例各异的数据集中自动筛选干净样本。训练过程中,OGRS方法在每次迭代时选择干净样本,并将选中的样本用于增量更新模型参数。我们通过引入并证明该非凸约束优化问题的次线性局部拉格朗日遗憾值,提供了详细的理论分析,证明数据选择过程会收敛到样本空间的低损失区域。实验结果表明,该方法在不同设置下的表现均优于现有最优方法。

英文摘要:

Learning with noisy labels is an important topic for scalable training in many real-world scenarios. However, few previous research considers this problem in the online setting, where the arrival of data is streaming. In this paper, we propose a novel gradient-based approach to enable the detection of noisy labels for the online learning of model parameters, named Online Gradient-based Robust Selection (OGRS). In contrast to the previous sample selection approach for the offline training that requires the estimation of a clean ratio of the dataset before each epoch of training, OGRS can automatically select clean samples by steps of gradient update from datasets with varying clean ratios without changing the parameter setting. During the training process, the OGRS method selects clean samples at each iteration and feeds the selected sample to incrementally update the model parameters. We provide a detailed theoretical analysis to demonstrate data selection process is converging to the low-loss region of the sample space, by introducing and proving the sub-linear local Lagrangian regret of the non-convex constrained optimization problem. Experimental results show that it outperforms state-of-the-art methods in different settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑