ManiEdit：从流形视角对语言模型进行顺序非结构化知识编辑
ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspective
浏览论文内容
中文总结 AI 辅助
ManiEdit从流形视角将知识编辑视为子流形局部位移，通过枢轴定位和流形感知保留，实现顺序非结构化知识编辑，在基准上取得最优性能并保持通用能力。
中文摘要 AI 辅助
大型语言模型（LLMs）不可避免地会生成一些不正确或过时的内容，因此需要高效且精确的机制来进行持续的知识更新。然而，现有的模型编辑方法难以顺序编辑非结构化的长形式知识，遭受严重的编辑遗忘和通用能力下降的问题。为了解决这些挑战，我们从流形视角重新构建知识编辑，将其视为全局知识流形中编辑子流形的局部位移。在这种表述下，问题可以分解为两个关键问题：（i）如何识别能够有效锚定编辑子流形的代表性编辑点，以及（ii）如何在子流形位移过程中保留剩余的流形结构。基于这一视角，我们提出了ManiEdit，一种新颖的流形感知自回归编辑框架，包含两个核心组件。枢轴定位解决了平庸点困境，通过识别高杠杆枢轴来锚定编辑子流形。流形感知保留通过能量加权惩罚结合递归零空间对齐来保留不同的知识类型。在两个基础LLM和四个非结构化编辑基准上的实验表明，ManiEdit实现了最先进的性能，在BERTScore上超过最强基线最多+27.81，在ROUGE-L上超过最多+8.50，同时在六个代表性下游任务中保持接近原始的通用能力。我们的代码可在以下网址获取：this https URL
英文摘要
Large language models (LLMs) inevitably generate some incorrect or outdated content, necessitating efficient and precise mechanisms for continual knowledge updates. However, existing model editing methods struggle to sequentially edit unstructured long-form knowledge, suffering from severe edit forgetting and degradation of general capabilities. To address these challenges, we reframe knowledge editing from a manifold perspective, viewing it as a localized displacement of an edit sub-manifold within the global knowledge manifold. Under this formulation, the problem can be decomposed into two key questions: (i) how to identify representative edit points that effectively anchor the edit sub-manifold, and (ii) how to preserve the remaining manifold structure during the sub-manifold displacement process. Based on this perspective, we propose ManiEdit, a novel manifold-aware autoregressive editing framework consisting of two core components. Pivot Localization addresses the mediocre-point dilemma by identifying high-leverage pivots to anchor the edit sub-manifold. Manifold-Aware Preservation preserves different knowledge types through an energy-weighted penalty combined with recursive null-space alignment. Experiments on two base LLMs and four unstructured editing benchmarks demonstrate that ManiEdit achieves state-of-the-art performance, outperforming the strongest baseline by up to +27.81 BERTScore and +8.50 ROUGE-L, while maintaining near-original general capabilities across six representative downstream tasks. Our code is available at: https://github.com/Areyliu/ManiEdit
发表机构
- Peking University（北京大学）
机构由 AI 辅助整理，请以论文原文为准。
