arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.00570cs.CLcs.AI
收藏

重新审视大型语言模型中基于参数的知识编辑:理论极限与实证证据

Revisiting Parameter-Based Knowledge Editing in Large Language Models: Theoretical Limits and Empirical Evidence

University of Science and Technology of China(中国科学技术大学)

Wanying Ren, Xin Song, Futing Wang, Guoxiu He, Aixin Sun

AI总结:

本文通过理论分析和实证评估,揭示了基于参数的知识编辑方法会因维度坍缩假设导致全局干扰和推理崩溃,而简单的检索基线方法在所有条件下均表现更优。

摘要:

基于参数的知识编辑通过局部权重修改更新大型语言模型(LLMs)的内部知识,并引起了广泛关注。然而,大多数现有方法忽略了基本的理论限制,并且很少在现实的、面向实践的设置下进行评估。在本文中,我们首先基于维度坍缩假设提出理论分析,解释局部参数编辑如何沿着表示空间中的脆弱方向传播,引发全局干扰并最终导致推理崩溃。基于这一见解,我们通过系统变化知识复杂度、编辑次数、评估维度和基线方法进行了全面的实证评估。我们的结果表明,基于参数的编辑方法持续损害LLM的核心能力。相比之下,一个简单的基于检索的基线在所有评估条件下始终比所有参数编辑方法表现更强。这些发现强调,在知识编辑后保持LLM的基本能力应成为未来研究的核心关注点。

英文摘要:

Parameter-based knowledge editing updates the internal knowledge of large language models (LLMs) via localized weight modifications and has attracted significant attention. However, most existing methods overlook fundamental theoretical limitations and are rarely evaluated under realistic, practice-oriented settings. In this paper, we first present a theoretical analysis based on the dimensional Collapse Hypothesis, explaining how localized parameter edits can propagate along fragile directions in the representation space, inducing global interference and ultimately causing reasoning collapse. Building on this insight, we conduct a comprehensive empirical evaluation by systematically varying knowledge complexity, number of edits, evaluation dimensions, and baseline methods. Our results show that parameter-based editing methods consistently damage core LLM capabilities. In contrast, a simple retrieval-based baseline achieves consistently stronger performance than all parameter-editing methods across all evaluated conditions. These findings highlight that preserving the fundamental capabilities of LLMs after knowledge editing should be a central concern for future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑