BetaEdit:基于空空间的顺序模型编辑
BetaEdit: Null-Space Constrained Sequential Model Editing
AI总结:
本文提出BetaEdit框架,通过控制知识泄漏并整合历史感知更新,提升大规模顺序编辑性能,实验显示其优于现有方法。
摘要:
基于空空间的方法通过约束更新到现有知识表示的空空间来保持模型原有行为,但在实践中依赖近似空空间导致知识泄漏,并在顺序编辑中表现下降。本文揭示现有空空间方法的知识泄漏问题,分析历史感知更新如何在长时间编辑中保持性能和通用能力。基于此,提出BetaEdit框架,有效控制知识泄漏并整合历史感知更新到空空间范式中。在两个标准基准上对三个大语言模型的大量实验表明,BetaEdit在大规模顺序编辑挑战中优于先前方法。代码可在https://github.com/lbq8942/BetaEdit获取。
英文摘要:
Null-space-based methods have garnered considerable attention in model editing by constraining updates to the null space of the pre-existing knowledge representation, thereby preserving the model's original behavior. However, in practice these methods rely on an approximate null space--leading to knowledge leakage--and further suffer from severe performance degradation during sequential editing. Recent work shows that history-aware editing strategies can empirically mitigate this decline, yet the underlying reason remains unclear. In this paper, we first expose the knowledge leakage inherent in existing null-space approaches and then analyze why history-aware updates effectively preserve both editing performance and general capabilities during long-horizon editing. Building on these insights, we propose BetaEdit, a refined framework that effectively controls the knowledge leakage and integrates history-aware updates into the null-space paradigm. Extensive experiments on three large language models across two standard benchmarks show that BetaEdit consistently outperforms prior methods in the challenging regime of massive-scale sequential editing. Code is available at: https://github.com/lbq8942/BetaEdit.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。