arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.01493cs.LGstat.ML
收藏

标签噪声及其他噪声下的指数梯度重加权鲁棒训练

Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond

University of California, Santa Cruz(加州大学圣克鲁兹分校);Google Research(谷歌研究院)

Negin Majidi, Ehsan Amid, Hossein Talebi, Manfred K. Warmuth

摘要:

机器学习中的许多学习任务可以被视为在每次训练迭代中采取梯度步骤,以最小化一批样本的平均损失。当噪声在数据中普遍存在时,这种对示例的统一处理可能导致对具有较大损失值的噪声示例的过拟合,并导致泛化能力差。受在线学习中专家设置的启发,我们提出了一种灵活的方法来从嘈杂的例子中学习。具体来说,我们将每个训练样本视为专家,并在所有样本上保持分布。我们交替使用梯度下降更新模型的参数和使用指数梯度更新示例权重。与其他相关方法不同,我们的方法处理一般类的损失函数,并可以应用于广泛的噪声类型和应用。我们展示了我们的方法对多个学习设置的有效性,即噪声主成分分析和各种噪声分类问题。

英文摘要:

Many learning tasks in machine learning can be viewed as taking a gradient step towards minimizing the average loss of a batch of examples in each training iteration. When noise is prevalent in the data, this uniform treatment of examples can lead to overfitting to noisy examples with larger loss values and result in poor generalization. Inspired by the expert setting in on-line learning, we present a flexible approach to learning from noisy examples. Specifically, we treat each training example as an expert and maintain a distribution over all examples. We alternate between updating the parameters of the model using gradient descent and updating the example weights using the exponentiated gradient update. Unlike other related methods, our approach handles a general class of loss functions and can be applied to a wide range of noise types and applications. We show the efficacy of our approach for multiple learning settings, namely noisy principal component analysis and a variety of noisy classification problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑