arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.13048cs.CL
收藏

大语言模型中的稳定知识编辑

Stable Knowledge Editing in Large Language Models

Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)

Zihao Wei, Liang Pang, Hanxing Ding, Jingcheng Deng, Huawei Shen, Xueqi Cheng

AI总结:

针对现有知识编辑方法因局部化和隔离假设导致的不稳定问题,本文提出StableKE,通过语义释义增强和上下文描述丰富两种自动化知识增强策略,在保证编辑效果的同时保留无关知识与通用能力,并可用于ChatGPT。

摘要:

大语言模型的高效知识编辑对于大规模替换过时信息或纳入专业知识至关重要。然而,以往方法隐含地假设知识在模型中是局部化且相互隔离的,这一假设过度简化了模型知识相互关联的本质。局部化的前提导致知识编辑不完整,而隔离假设则可能损害其他知识和通用能力,从而给知识编辑方法的性能带来不稳定性。为了超越这些假设,我们提出了StableKE,该方法采用基于知识增强而非知识局部化的新视角。为克服人工标注的高昂成本,StableKE集成了两种自动化知识增强策略:语义释义增强策略,通过多样化知识描述以促进向模型传授新信息;以及上下文描述丰富策略,通过扩展周围知识以防止遗忘相关信息。StableKE超越了其他知识编辑方法,在编辑知识和多跳知识方面均表现出稳定性,同时还能保留无关知识和通用能力。此外,StableKE可以在ChatGPT上编辑知识。

英文摘要:

Efficient knowledge editing of large language models is crucial for replacing obsolete information or incorporating specialized knowledge on a large scale. However, previous methods implicitly assume that knowledge is localized and isolated within the model, an assumption that oversimplifies the interconnected nature of model knowledge. The premise of localization results in an incomplete knowledge editing, whereas an isolated assumption may impair both other knowledge and general abilities. It introduces instability to the performance of the knowledge editing method. To transcend these assumptions, we introduce StableKE, a method adopts a novel perspective based on knowledge augmentation rather than knowledge localization. To overcome the expense of human labeling, StableKE integrates two automated knowledge augmentation strategies: Semantic Paraphrase Enhancement strategy, which diversifies knowledge descriptions to facilitate the teaching of new information to the model, and Contextual Description Enrichment strategy, expanding the surrounding knowledge to prevent the forgetting of related information. StableKE surpasses other knowledge editing methods, demonstrating stability both edited knowledge and multi-hop knowledge, while also preserving unrelated knowledge and general abilities. Moreover, StableKE can edit knowledge on ChatGPT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑