arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.16401cs.LGcs.ITmath.IT
收藏

标签噪声下学习用的Fisher-Rao损失

The Fisher-Rao Loss for Learning under Label Noise

Institute of Mathematics, Statistics and Scientific Computing (IMECC), University of Campinas (Unicamp)(坎皮纳斯大学数学、统计与科学计算研究所)

Henrique K. Miyamoto, Fábio C. C. Meneghetti, Sueli I. R. Costa

AI总结:

针对标签噪声下的分类器训练问题,研究源自离散分布统计流形Fisher-Rao距离的Fisher-Rao损失,推导其性能下降上界并分析学习速度,指出该损失在鲁棒性与训练动态间实现自然权衡,经合成数据和MNIST实验验证性能。

摘要:

在通过经验风险最小化进行学习时,选择合适的损失函数至关重要。在许多实际场景中,用于训练分类器的数据集可能包含错误标签,这促使人们关注使用本身对标签噪声具有鲁棒性的损失函数。本文研究了Fisher-Rao损失函数,它源自离散分布统计流形上的Fisher-Rao距离。我们推导了存在标签噪声时性能下降的上界,并分析了该损失的学习速度。与其他常用损失相比,我们认为Fisher-Rao损失在鲁棒性和训练动态之间实现了自然的权衡。在合成数据集和MNIST数据集上的数值实验验证了这一性能表现。

英文摘要:

Choosing a suitable loss function is essential when learning by empirical risk minimisation. In many practical cases, the datasets used for training a classifier may contain incorrect labels, which prompts the interest for using loss functions that are inherently robust to label noise. In this paper, we study the Fisher-Rao loss function, which emerges from the Fisher-Rao distance in the statistical manifold of discrete distributions. We derive an upper bound for the performance degradation in the presence of label noise, and analyse the learning speed of this loss. Comparing with other commonly used losses, we argue that the Fisher-Rao loss provides a natural trade-off between robustness and training dynamics. Numerical experiments with synthetic and MNIST datasets illustrate this performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑