arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.11849stat.MLcs.LG
收藏

深度网络在比样本更精细的层次上的选择性遗忘

Selective Forgetting of Deep Networks at a Finer Level than Samples

Fujitsu Laboratories Ltd.(富士通实验室有限公司)

Tomohiro Hayase, Suguru Yasutomi, Takashi Katoh

摘要:

选择性遗忘或从深度神经网络(DNN)中删除信息对于持续学习至关重要,并且在控制DNN方面具有挑战性。这种遗忘在实际意义上也是至关重要的,因为部署的DNN可能是在具有离群值的数据上训练的,被攻击者毒害,或具有泄露/敏感信息。在本文中,我们制定了选择性遗忘的分类任务在一个更精细的水平比样本的水平。我们基于四个数据集指定更精细的级别,这四个数据集由两个条件区分:它们是否包含要遗忘的信息以及它们是否可用于遗忘过程。此外,我们揭示了需要这样的配方与数据集显示具体和实际的情况。此外,我们引入遗忘过程作为一个优化问题的三个标准:遗忘,校正,和记忆项。实验结果表明,该方法可以使模型忘记使用特定的信息进行分类。值得注意的是,在特定情况下,我们的方法提高了模型在数据集上的准确性,这些数据集包含要遗忘的信息,但在遗忘过程中不可用。这些数据在实际情况中意外发现并被错误分类。

英文摘要:

Selective forgetting or removing information from deep neural networks (DNNs) is essential for continual learning and is challenging in controlling the DNNs. Such forgetting is crucial also in a practical sense since the deployed DNNs may be trained on the data with outliers, poisoned by attackers, or with leaked/sensitive information. In this paper, we formulate selective forgetting for classification tasks at a finer level than the samples' level. We specify the finer level based on four datasets distinguished by two conditions: whether they contain information to be forgotten and whether they are available for the forgetting procedure. Additionally, we reveal the need for such formulation with the datasets by showing concrete and practical situations. Moreover, we introduce the forgetting procedure as an optimization problem on three criteria; the forgetting, the correction, and the remembering term. Experimental results show that the proposed methods can make the model forget to use specific information for classification. Notably, in specific cases, our methods improved the model's accuracy on the datasets, which contains information to be forgotten but is unavailable in the forgetting procedure. Such data are unexpectedly found and misclassified in actual situations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑