ALOE：用于知识编辑的语义寻址低秩算子

中文摘要 AI 辅助

知识编辑通过修改参数来改变模型所知的内容，使得所请求的事实得到更新，同时不相关的行为得以保留。这通常被视为一个写入问题，但编辑还涉及一个寻址问题：决定哪些隐藏状态应接收新的残差。激活范围过窄的更新会记忆单个提示，而激活范围过宽的更新则会干扰邻近知识。参数化编辑器隐式地编码了这种作用范围，而基于记忆的编辑器则使选择显式化，但将其保留在被编辑模型之外。我们提出ALOE（寻址低秩编辑算子），它从释义和困难同主题负样本中学习语义地址，通过滚动细化（rollout refinement）和门控校准（gate calibration）将这些地址与自回归隐藏状态对齐，并将由此产生的门控低秩算子嵌入到一个MLP层中，从而使部署后的模型在单次前向传播中运行，无需外部检索器或辅助路由器。在CounterFact、ZSRE和KnowEdit三个7B至8B模型家族上的评估中，ALOE的有效性介于0.955至0.999之间，局部性介于0.981至1.000之间；机制分析证实，学习到的几何结构能分离相互竞争的编辑，且校准能抑制超出作用范围的激活。剩余的误差集中在释义覆盖和写入拟合方面。

英文摘要

Knowledge editing changes what a model knows by modifying parameters so that a requested fact updates while unrelated behavior is preserved. This is usually treated as a write problem, but editing also involves an address problem: deciding which hidden states should receive the new residual. An update that activates too narrowly memorizes one prompt, while one that activates too broadly disrupts neighboring knowledge. Parametric editors encode this scope implicitly, whereas memory-based editors make the selection explicit but keep it outside the edited model. We propose ALOE (Addressed Low-rank Operator for Editing), which learns semantic addresses from paraphrases and hard same-subject negatives, aligns them with autoregressive hidden states through rollout refinement and gate calibration, and embeds the resulting gated low-rank operator within one MLP layer, so that the deployed model runs in a single forward pass with no external retriever or auxiliary router. Evaluated on CounterFact, ZSRE, and KnowEdit across three 7--8B model families, ALOE achieves efficacy between 0.955 and 0.999 and locality between 0.981 and 1.000; mechanistic analyses confirm that the learned geometry separates competing edits and that calibration suppresses out-of-scope activation. The remaining errors concentrate in paraphrase coverage and write fitting.

发表机构

• Shanghai Jiao Tong University（上海交通大学）

机构由 AI 辅助整理，请以论文原文为准。