arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.01670cs.LG
收藏

鲁棒神经过程用于噪声数据

Robust Neural Processes for Noisy Data

University of Haifa(海法大学)

Chen Shapira, Dan Rosenbaum

AI总结:

针对噪声数据下上下文学习模型性能下降问题,提出基于神经过程的鲁棒训练方法,实验证明其在各噪声水平下均优于现有NP模型。

摘要:

基于给定上下文调整预测的模型,也称为上下文学习,近年来已变得无处不在。我们提出研究当数据被噪声污染时此类模型的行为。为此,我们使用神经过程(NP)框架,作为一种简单且严谨的学习函数分布的方法,其中预测基于一组上下文点。使用该框架,我们发现,在干净数据上表现最佳的模型与在噪声数据上表现最佳的模型是不同的。具体而言,使用注意力处理上下文的模型受噪声影响更为严重,导致上下文过拟合。我们提出一种简单的训练NP模型的方法,使其对噪声数据更加鲁棒。在1D函数和2D图像数据集上的实验表明,我们的方法使模型在所有噪声水平下均优于所有其他NP模型。

英文摘要:

Models that adapt their predictions based on some given contexts, also known as in-context learning, have become ubiquitous in recent years. We propose to study the behavior of such models when data is contaminated by noise. Towards this goal we use the Neural Processes (NP) framework, as a simple and rigorous way to learn a distribution over functions, where predictions are based on a set of context points. Using this framework, we find that the models that perform best on clean data, are different than the models that perform best on noisy data. Specifically, models that process the context using attention, are more severely affected by noise, leading to in-context overfitting. We propose a simple method to train NP models that makes them more robust to noisy data. Experiments on 1D functions and 2D image datasets demonstrate that our method leads to models that outperform all other NP models for all noise levels.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑