通过自构知识三元组实现大语言模型中的概念遗忘
Concept Unlearning in Large Language Models via Self-Constructed Knowledge Triplets
AI总结：
本文提出概念遗忘（CU）任务，通过让LLM自构知识三元组并与内部知识表示对齐，实现人物、事件等概念级遗忘，同时保留无关知识。
摘要：
机器遗忘（Machine Unlearning，MU）作为解决大语言模型（LLMs）中隐私和版权问题的一种方案，近期引起了广泛关注。现有的MU方法旨在从LLM中移除特定的目标句子，同时尽量减少对无关知识的损害。然而，这些方法需要明确的目标句子，并且不支持移除更广泛的概念，例如人物或事件。为了解决这一局限，我们提出概念遗忘（Concept Unlearning，CU）作为LLM遗忘的一个新需求。我们利用知识图谱来表示LLM的内部知识，并将CU定义为移除遗忘目标节点及其关联边。这种基于图的表述使得遗忘过程更加直观，并有助于设计更有效的方法。我们提出了一种新方法，该方法提示LLM生成关于遗忘目标的知识三元组和解释性句子，并将遗忘过程应用于这些表示。通过将遗忘过程与LLM的内部知识表示对齐，我们的方法能够实现更精确、更全面的概念移除。在真实世界和合成数据集上的实验表明，我们的方法能够有效地实现概念级别的遗忘，同时保留无关知识。
英文摘要：
Machine Unlearning (MU) has recently attracted considerable attention as a solution to privacy and copyright issues in large language models (LLMs). Existing MU methods aim to remove specific target sentences from an LLM while minimizing damage to unrelated knowledge. However, these approaches require explicit target sentences and do not support removing broader concepts, such as persons or events. To address this limitation, we introduce Concept Unlearning (CU) as a new requirement for LLM unlearning. We leverage knowledge graphs to represent the LLM's internal knowledge and define CU as removing the forgetting target nodes and associated edges. This graph-based formulation enables a more intuitive unlearning and facilitates the design of more effective methods. We propose a novel method that prompts the LLM to generate knowledge triplets and explanatory sentences about the forgetting target and applies the unlearning process to these representations. Our approach enables more precise and comprehensive concept removal by aligning the unlearning process with the LLM's internal knowledge representations. Experiments on real-world and synthetic datasets demonstrate that our method effectively achieves concept-level unlearning while preserving unrelated knowledge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
