arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.06535cs.CL
收藏

KGA:一种基于知识差距对齐的通用机器遗忘框架

KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment

The Chinese University of Hong Kong(香港中文大学);MoE Key Laboratory of High Confidence Software Technologies(教育部高可信软件技术重点实验室);The University of Queensland(昆士兰大学)

Lingzhi Wang, Tong Chen, Wei Yuan, Xingshan Zeng, Kam-Fai Wong, Hongzhi Yin

AI总结:

本文提出基于知识差距对齐的通用机器遗忘框架KGA,首次应用于多种NLP任务,通过保持分布差异实现遗忘,并在大规模数据集上取得全面改进。

摘要:

近期颁布的“被遗忘权”相关法律引发了人们对机器遗忘的兴趣,即让已学习的模型具备遗忘特定训练实例信息的功能,仿佛这些实例从未出现在训练集中。以往的工作主要关注计算机视觉场景,在很大程度上忽略了自然语言处理领域中遗忘的本质,而文本数据比图像包含更明确和敏感的个人信息。在本文中,我们提出了一种名为KGA的通用遗忘框架,以引发遗忘。与以往试图恢复梯度或迫使模型表现接近某一特定分布的工作不同,KGA保持分布差异(即知识差距)。这放宽了分布假设。此外,我们首次将遗忘方法应用于各种NLP任务(即分类、翻译、回复生成),并提出了若干具有针对性的遗忘评估指标。在大型数据集上的实验表明,KGA在基线上取得了全面的改进,广泛的分析进一步验证了KGA的有效性,并为NLP任务的遗忘提供了见解。

英文摘要:

Recent legislation of the "right to be forgotten" has led to the interest in machine unlearning, where the learned models are endowed with the function to forget information about specific training instances as if they have never existed in the training set. Previous work mainly focuses on computer vision scenarios and largely ignores the essentials of unlearning in NLP field, where text data contains more explicit and sensitive personal information than images. In this paper, we propose a general unlearning framework called KGA to induce forgetfulness. Different from previous work that tries to recover gradients or forces models to perform close to one specific distribution, KGA maintains distribution differences (i.e., knowledge gap). This relaxes the distribution assumption. Furthermore, we first apply the unlearning method to various NLP tasks (i.e., classification, translation, response generation) and propose several unlearning evaluation metrics with pertinence. Experiments on large-scale datasets show that KGA yields comprehensive improvements over baselines, where extensive analyses further validate the effectiveness of KGA and provide insight into unlearning for NLP tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑