AI 中文总结
本研究针对多层知识编辑方法存在的传播衰减问题,提出无锚点的DOW-KE方法,联合优化所有编辑层权重更新,在多数模型-数据集设置中取得最优性能。
AI 中文摘要
用于知识编辑的多层定位后编辑方法,首先在选定层优化目标残差流激活(锚点),再逐层将其实现为权重更新。该流程优化了中间表示,但部署的是多层权重更新,其通过真实前向传播的联合效果从未被优化:无论锚点如何设置或传播,每次更新均来自局部求解,因此传播诱导的衰减和失真未被修正,在锚点目标与实现的编辑之间留下了闭合差距。我们提出DOW-KE,一种基于单一原则的无锚点方法:被优化的必须恰好是被部署的。DOW-KE通过完整模型反向传播最终编辑目标,联合优化所有编辑层的更新,使跨层传播和耦合进入每一步梯度。相同原则决定了保留的位置:在计算图内部的更新参数化中嵌入保留投影,使每一步梯度作用于部署的更新;事后约束会重新打开差距,而受约束的搜索可使编辑避开受保护的知识。在两个数据集和三个模型上的大规模顺序编辑实验中,DOW-KE在评估基准的6种模型-数据集设置中,5种实现了最高的总体得分和邻域特异性。
英文摘要
Multi-layer locate-then-edit methods for knowledge editing first optimize target residual-stream activations (anchors) at selected layers, then realize them layer by layer as weight updates. This pipeline optimizes an intermediate representation but deploys multi-layer weight updates whose joint effect through the true forward pass is never itself optimized: regardless of how anchors are set or propagated, each update comes from a local solve, so propagation-induced attenuation and distortion go uncorrected, leaving a closure gap between anchor targets and realized edits. We propose DOW-KE, an anchor-free method built on a single principle: what is optimized must be exactly what is deployed. DOW-KE backpropagates the final editing objective through the complete model, jointly optimizing the updates of all edited layers so cross-layer propagation and coupling enter every gradient step. The same principle dictates where preservation resides: embedding the preservation projection in the update parameterization, inside the computation graph, makes every gradient act on the deployed update; post-hoc constraints would reopen the gap, and the constrained search keeps edits clear of protected knowledge. In large-scale sequential editing on two datasets and three models, DOW-KE achieves the highest overall Score and neighborhood Specificity in five of six model-dataset settings among the evaluated baselines.