为什么新知识在大语言模型中会产生混乱的涟漪效应?
Why Does New Knowledge Create Messy Ripple Effects in LLMs?
AI总结:
本文提出GradSim指标,通过原始事实与相关知识梯度的余弦相似度解释并预测大语言模型中知识编辑涟漪效应的成败,并在多种模型和方法上验证其有效性。
摘要:
大量先前研究聚焦于语言模型(LMs)的post-training knowledge editing(KE,训练后知识编辑),以确保知识保持准确和最新。KE中一个期望的性质和开放问题是让编辑后的LMs正确处理涟漪效应,即期望LM能够准确回答其逻辑上相关的知识。在本文中,我们回答了为什么大多数KE方法仍然产生混乱的涟漪效应这一问题。我们进行了广泛的分析,并识别出一个显著的指标GradSim,它有效地揭示了更新后的知识何时以及为何在LMs中产生涟漪。GradSim通过原始事实与其相关知识之间梯度的余弦相似度来计算。我们观察到,在不同的LMs、KE方法和评估指标上,涟漪效应性能与GradSim之间存在强正相关。对涟漪效应的三个反直觉失败案例(Negation、Over-Ripple、Multi-Lingual)的进一步研究表明,这些失败通常与非常低的GradSim相关。这一发现验证了GradSim是知识何时在LMs中产生涟漪的有效指标。
英文摘要:
Extensive previous research has focused on post-training knowledge editing (KE) for language models (LMs) to ensure that knowledge remains accurate and up-to-date. One desired property and open question in KE is to let edited LMs correctly handle ripple effects, where LM is expected to answer its logically related knowledge accurately. In this paper, we answer the question of why most KE methods still create messy ripple effects. We conduct extensive analysis and identify a salient indicator, GradSim, that effectively reveals when and why updated knowledge ripples in LMs. GradSim is computed by the cosine similarity between gradients of the original fact and its related knowledge. We observe a strong positive correlation between ripple effect performance and GradSim across different LMs, KE methods, and evaluation metrics. Further investigations into three counter-intuitive failure cases (Negation, Over-Ripple, Multi-Lingual) of ripple effects demonstrate that these failures are often associated with very low GradSim. This finding validates that GradSim is an effective indicator of when knowledge ripples in LMs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。