arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.01944cs.LGcs.AI
收藏

基于特征爬虫的带采样噪声和标签噪声的鲁棒元学习

Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile

JD.COM Silicon Valley Research Center(京东硅谷研究中心)

Dong Chen, Lingfei Wu, Siliang Tang, Xiao Yun, Bo Long, Yueting Zhuang

摘要:

近年来,人们对用于解决少样本学习(FSL)问题的元学习技术的兴趣激增。然而,元学习器容易过拟合,因为只有少数可用的样本,这可以被识别为干净数据集上的采样噪声。此外,当处理带有噪声标签的数据时,元学习器可能对损坏数据集上的标签噪声非常敏感。为了解决这两个挑战,我们提出了本征爬虫(ER),更新的元参数与历史任务特定的参数,以减轻采样和标签噪声的主要方向。具体地,以快速的方式计算主方向,其中所计算的矩阵的尺度与梯度步长的数量而不是参数的数量相关。此外,为了在存在许多噪声标签的情况下获得更准确的特征爬行动物主方向,我们进一步提出了内省自步调学习(ISPL)。我们已经从理论和实验上证明了所提出的本征爬行动物和ISPL的合理性和有效性。特别是,我们对不同任务的实验表明,该方法能够优于或实现具有高度竞争力的性能相比,其他基于梯度的方法有或没有噪声标签。所提出的方法的代码和数据提供用于研究目的https://github.com/Anfeather/Eigen-Reptile。

英文摘要:

Recent years have seen a surge of interest in meta-learning techniques for tackling the few-shot learning (FSL) problem. However, the meta-learner is prone to overfitting since there are only a few available samples, which can be identified as sampling noise on a clean dataset. Moreover, when handling the data with noisy labels, the meta-learner could be extremely sensitive to label noise on a corrupted dataset. To address these two challenges, we present Eigen-Reptile (ER) that updates the meta-parameters with the main direction of historical task-specific parameters to alleviate sampling and label noise. Specifically, the main direction is computed in a fast way, where the scale of the calculated matrix is related to the number of gradient steps instead of the number of parameters. Furthermore, to obtain a more accurate main direction for Eigen-Reptile in the presence of many noisy labels, we further propose Introspective Self-paced Learning (ISPL). We have theoretically and experimentally demonstrated the soundness and effectiveness of the proposed Eigen-Reptile and ISPL. Particularly, our experiments on different tasks show that the proposed method is able to outperform or achieve highly competitive performance compared with other gradient-based methods with or without noisy labels. The code and data for the proposed method are provided for research purposes https://github.com/Anfeather/Eigen-Reptile.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑