arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.00846stat.MLcs.LG
收藏

PUMA:用于训练数据删除的性能不变模型增强

PUMA: Performance Unchanged Model Augmentation for Training Data Removal

Borealis AI

Ga Wu, Masoud Hashemi, Christopher Srinivasa

摘要:

保留训练模型的性能,同时去除标记的训练数据点的独特特征是具有挑战性的。最近的研究通常建议使用剩余的训练数据从头开始重新训练模型,或者通过在标记的数据点上恢复模型优化来改进模型。不幸的是,除了计算效率低之外,这些方法不可避免地损害了结果模型的泛化能力,因为它们不仅删除了唯一的特征,而且还丢弃了共享的(可能是贡献的)信息。为了解决性能下降的问题,本文提出了一种新的方法称为性能不变的模型增强~(PEM-A)。所提出的CNOAA框架明确地模拟每个训练数据点对模型的泛化能力的影响,相对于各种性能标准。然后,它通过最佳地重新加权剩余数据来补充删除标记数据的负面影响。为了证明PUMA框架的有效性,我们在实验中将其与多种最先进的数据删除技术进行了比较,其中我们表明PUMA可以有效且高效地删除标记训练数据的独特特征,而无需重新训练模型1)欺骗成员攻击,2)抵抗性能下降。此外,由于CNOAA在其操作过程中估计数据的重要性,我们表明它可以比现有方法更有效地调试错误标记的数据点。

英文摘要:

Preserving the performance of a trained model while removing unique characteristics of marked training data points is challenging. Recent research usually suggests retraining a model from scratch with remaining training data or refining the model by reverting the model optimization on the marked data points. Unfortunately, aside from their computational inefficiency, those approaches inevitably hurt the resulting model's generalization ability since they remove not only unique characteristics but also discard shared (and possibly contributive) information. To address the performance degradation problem, this paper presents a novel approach called Performance Unchanged Model Augmentation~(PUMA). The proposed PUMA framework explicitly models the influence of each training data point on the model's generalization ability with respect to various performance criteria. It then complements the negative impact of removing marked data by reweighting the remaining data optimally. To demonstrate the effectiveness of the PUMA framework, we compared it with multiple state-of-the-art data removal techniques in the experiments, where we show the PUMA can effectively and efficiently remove the unique characteristics of marked training data without retraining the model that can 1) fool a membership attack, and 2) resist performance degradation. In addition, as PUMA estimates the data importance during its operation, we show it could serve to debug mislabelled data points more efficiently than existing approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑