KLOD：通过非目标分布保持实现的局部性保留型知识编辑
KLOD: Locality-Preserving Knowledge Editing via Non-Target Distribution Preservation
- Gachon University（嘉泉大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
KLOD是一种用于微调知识编辑的有界分布保持目标函数，通过保留特定位置的输出分布减轻局部性下降，在CounterFact等数据集上针对Llama3等模型验证了其效果，还可控制泛化-局部性权衡。
AI中文摘要：
基于微调的知识编辑方法简单且与架构无关，但标准交叉熵会提高编辑目标概率，却未明确约束非目标输出分布的变化。在顺序编辑中，这种无约束的分布重分配会累积为分布漂移，导致局部性下降。我们提出KLOD，一种用于基于微调的知识编辑的有界且分布保持的目标函数，它将预期的目标更新与应保持稳定的分布分离开。KLOD在达到概率阈值后会停止目标放大，同时保留目标位置处排除目标后的非目标分布，以及前缀位置处的完整下一个token分布。在CounterFact和ZsRE数据集上针对Llama3-8B-Instruct和Qwen2.5-7B-Instruct模型的实验表明，KLOD在保持高编辑可靠性的同时，大幅减轻了局部性下降。目标概率阈值还提供了可控制的泛化-局部性权衡。消融实验、多随机种子实验和分布KL分析支持以下解释：KLOD的局部性提升与输出分布的保留相关，而非单纯削弱编辑效果。代码可在GitHub获取：this https URL。
英文摘要：
Fine-tuning-based knowledge editing is simple and architecture-agnostic, but standard cross-entropy increases the edited target probability without explicitly constraining changes in the non-target output distribution. In sequential editing, such unconstrained redistribution can accumulate as distributional drift and contribute to locality degradation. We propose KLOD, a bounded and distribution-preserving objective for fine-tuning-based knowledge editing that separates the intended target update from distributions that should remain stable. KLOD stops target amplification once a probability threshold is reached, while preserving the target-excluded non-target distribution at target positions and the full next-token distribution at prefix positions. Experiments on CounterFact and ZsRE with Llama3-8B-Instruct and Qwen2.5-7B-Instruct show that KLOD substantially mitigates locality degradation while maintaining high edit reliability. The target probability threshold further provides a controllable Generalization--Locality trade-off. Ablation, multi-seed, and distributional KL analyses support the interpretation that KLOD's locality gains are associated with preserving output distributions rather than simply weakening the edit. Code is available on GitHub https://github.com/Hostoday/KLOD .
