arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.06259cs.CLcs.AIcs.DBcs.IRcs.LG
收藏

面向大语言模型的概念性知识编辑

Editing Conceptual Knowledge for Large Language Models

Zhejiang University(浙江大学);ZJU-Ant Group Joint Research Center for Knowledge Graphs(浙江大学-蚂蚁集团知识图谱联合研究中心);National University of Singapore(新加坡国立大学);Ant Group(蚂蚁集团)

Xiaohan Wang, Shengyu Mao, Ningyu Zhang, Shumin Deng, Yunzhi Yao, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen

AI总结:

针对大语言模型知识编辑仅聚焦实例级、概念编辑能力不明的问题,该研究构建ConceptEdit基准数据集与新评估指标,发现现有方法虽可修改概念定义但会扭曲相关实例知识,为理解LLM提供新方向。

摘要:

近年来,大语言模型(LLM)的知识编辑研究受到越来越多的关注。现有方法与评估仅围绕实例级编辑展开,而LLM是否具备修改概念的能力仍不明确。本文率先开展LLM概念性知识编辑的研究,构建了全新的基准数据集ConceptEdit,并建立了一套新的评估指标。实验结果表明,尽管现有编辑方法能在一定程度上有效修改概念层面的定义,但它们也可能扭曲LLM中相关的实例性知识,导致性能不佳。我们期望本研究能推动对LLM更深入理解的后续进展。项目主页为:https://zjunlp.github.io/project/ConceptEdit。

英文摘要:

Recently, there has been a growing interest in knowledge editing for Large Language Models (LLMs). Current approaches and evaluations merely explore the instance-level editing, while whether LLMs possess the capability to modify concepts remains unclear. This paper pioneers the investigation of editing conceptual knowledge for LLMs, by constructing a novel benchmark dataset ConceptEdit and establishing a suite of new metrics for evaluation. The experimental results reveal that, although existing editing methods can efficiently modify concept-level definition to some extent, they also have the potential to distort the related instantial knowledge in LLMs, leading to poor performance. We anticipate this can inspire further progress in better understanding LLMs. Our project homepage is available at https://zjunlp.github.io/project/ConceptEdit.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑