arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.10098cs.LG
收藏

基于自适应选择性突触抑制的无参数调优数据录入错误遗忘

Parameter-tuning-free data entry error unlearning with adaptive selective synaptic dampening

University of Cambridge(剑桥大学);The Alan Turing Institute(阿兰·图灵研究所)

Stefan Schoepf, Jack Foster, Alexandra Brintrup

AI总结:

针对未知正确标签的数据录入错误,提出无需参数调优的自适应选择性突触抑制(ASSD)遗忘方法,在通用基准和供应链预测中表现优异。

摘要:

数据录入是机器学习流程的基本组成部分,但经常导致标签错误的引入。当模型在包含此类错误的数据集上训练时,其性能会下降。这带来了一个挑战:如何高效地遗忘错误数据的影响,以在不完全重新训练模型的情况下提升模型性能。虽然针对错误条目的正确标签已知的情况存在模型编辑方法,但我们关注的是不知道错误数据正确标签的数据录入错误情况。我们的贡献有两方面。首先,我们引入了对选择性突触抑制遗忘方法的扩展,该扩展消除了参数调优的需求,使从业者能够使用遗忘技术。我们展示了这一扩展,即自适应选择性突触抑制(ASSD),在各种ResNet18和Vision Transformer遗忘任务上的性能。其次,我们在一个带有标签错误的供应链延迟预测问题中展示了ASSD的性能,该问题使用了真实世界数据,我们在其中随机引入了不同级别的标签错误。这种方法的应用在工业场景(如供应链管理)中尤为引人注目,因为在这些场景中,大部分数据录入是通过Excel表格手动完成的,容易出错。ASSD在通用遗忘基准和错误纠正问题上表现出色,在错误纠正方面优于微调方法。

英文摘要:

Data entry constitutes a fundamental component of the machine learning pipeline, yet it frequently results in the introduction of labelling errors. When a model has been trained on a dataset containing such errors its performance is reduced. This leads to the challenge of efficiently unlearning the influence of the erroneous data to improve the model performance without needing to completely retrain the model. While model editing methods exist for cases in which the correct label for a wrong entry is known, we focus on the case of data entry errors where we do not know the correct labels for the erroneous data. Our contribution is twofold. First, we introduce an extension to the selective synaptic dampening unlearning method that removes the need for parameter tuning, making unlearning accessible to practitioners. We demonstrate the performance of this extension, adaptive selective synaptic dampening (ASSD), on various ResNet18 and Vision Transformer unlearning tasks. Second, we demonstrate the performance of ASSD in a supply chain delay prediction problem with labelling errors using real-world data where we randomly introduce various levels of labelling errors. The application of this approach is particularly compelling in industrial settings, such as supply chain management, where a significant portion of data entry occurs manually through Excel sheets, rendering it error-prone. ASSD shows strong performance on general unlearning benchmarks and on the error correction problem where it outperforms fine-tuning for error correction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑