$μ$KE：大型语言模型的Matryoshka非结构化知识编辑
$μ$KE: Matryoshka Unstructured Knowledge Editing of Large Language Models
AI总结：
针对LLM非结构化知识编辑破坏因果依赖的问题，提出$μ$KE机制，通过Matryoshka风格目标与自适应损失系数保持依赖，在四个基准上较SOTA提升编辑效力达12.33%。
摘要：
大型语言模型（LLMs）已成为强大的知识库，但受限于静态训练数据，导致幻觉和安全风险等问题。通过定位与编辑范式编辑模型内部知识已被证明是重训练的高性价比替代方案，然而当前的非结构化方法，尤其是基于窗口的自回归方法，常破坏早期记忆更新与后续输出词元间的因果依赖。本研究首先理论分析了这些局限，随后提出Matryoshka非结构化知识编辑（$μ$KE），这是一种通过Matryoshka风格目标函数和自适应损失系数来保持此类依赖的新型记忆更新机制。在四个基准上对两个模型的实证评估表明，$μ$KE较最先进方法将编辑效力提升达12.33%，且在应用于多种格式化编辑时保持稳健，凸显了其在LLMs中有效进行非结构化知识编辑的潜力。
英文摘要：
Large language models (LLMs) have emerged as powerful knowledge bases yet are limited by static training data, leading to issues such as hallucinations and safety risks. Editing a model's internal knowledge through the locate-and-edit paradigm has proven a cost-effective alternative to retraining, though current unstructured approaches, especially window-based autoregressive methods, often disrupt the causal dependency between early memory updates and later output tokens. In this work, we first theoretically analyze these limitations and then introduce Matryoshka Unstructured Knowledge Editing ($μ$KE), a novel memory update mechanism that preserves such dependencies via a Matryoshka-style objective and adaptive loss coefficients. Empirical evaluations on two models across four benchmarks demonstrate that $μ$KE improves edit efficacy by up to 12.33% over state-of-the-art methods, and remains robust when applied to diverse formatted edits, underscoring its potential for effective unstructured knowledge editing in LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
