揭示大型语言模型知识编辑的陷阱
Unveiling the Pitfalls of Knowledge Editing for Large Language Models
AI总结：
本文研究LLM知识编辑的潜在风险，构建新基准与评估指标，揭示知识冲突和知识扭曲两类陷阱。
摘要：
随着微调大型语言模型（LLMs）的相关成本持续上升，近期研究工作已转向开发用于编辑LLMs中内嵌隐式知识的方法。然而，仍有一片阴云笼罩——知识编辑是否会引发蝴蝶效应？因为目前尚不清楚知识编辑是否可能带来构成潜在风险的副作用。本文率先研究LLMs知识编辑相关的潜在陷阱。为此，我们引入了新的基准数据集，并提出了创新的评估指标。我们的结果强调了两个关键问题：（1）知识冲突：编辑在逻辑上相互冲突的成组事实，可能放大LLMs中固有的不一致性，这是以往方法所忽视的一个方面。（2）知识扭曲：为编辑事实性知识而改变参数，可能不可逆转地扭曲LLMs的固有知识结构。实验结果生动表明，知识编辑可能无意中给LLMs投下非预期后果的阴影，这值得未来工作予以关注和投入。代码和数据可在 https://github.com/zjunlp/PitfallsKnowledgeEditing 获取。
英文摘要：
As the cost associated with fine-tuning Large Language Models (LLMs) continues to rise, recent research efforts have pivoted towards developing methodologies to edit implicit knowledge embedded within LLMs. Yet, there's still a dark cloud lingering overhead -- will knowledge editing trigger butterfly effect? since it is still unclear whether knowledge editing might introduce side effects that pose potential risks or not. This paper pioneers the investigation into the potential pitfalls associated with knowledge editing for LLMs. To achieve this, we introduce new benchmark datasets and propose innovative evaluation metrics. Our results underline two pivotal concerns: (1) Knowledge Conflict: Editing groups of facts that logically clash can magnify the inherent inconsistencies in LLMs-a facet neglected by previous methods. (2) Knowledge Distortion: Altering parameters with the aim of editing factual knowledge can irrevocably warp the innate knowledge structure of LLMs. Experimental results vividly demonstrate that knowledge editing might inadvertently cast a shadow of unintended consequences on LLMs, which warrant attention and efforts for future works. Code and data are available at https://github.com/zjunlp/PitfallsKnowledgeEditing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
