迈向可扩展的终身知识编辑与选择性知识抑制
Towards Scalable Lifelong Knowledge Editing with Selective Knowledge Suppression
AI总结：
本文提出LightEdit方法，通过选择性知识抑制实现高效终身知识编辑，实验表明其在多个基准测试中表现优异，且降低了训练成本，提升了可扩展性。
摘要：
大型语言模型（LLMs）需要频繁更新知识以反映变化的事实并减少幻觉。为满足这一需求，终身知识编辑作为一种连续方法，允许在不重新训练整个模型的情况下修改特定知识。现有参数编辑方法在连续编辑中面临稳定性问题，而基于检索的方法由于高训练成本在不同数据集上应用受限。为解决这些限制并增强终身设置的可扩展性，我们提出LightEdit。我们的框架首先从检索信息中选择相关知识以有效修改查询，然后采用解码策略抑制模型原有知识的概率，从而基于所选信息实现高效编辑。在ZSRE、Counterfact和RIPE基准测试中的广泛实验表明，LightEdit优于现有终身知识编辑方法。此外，通过最小化训练成本，LightEdit实现了成本效益的可扩展性，使其能够轻松适应各种数据集。
英文摘要：
Large language models (LLMs) require frequent knowledge updates to reflect changing facts and mitigate hallucinations. To meet this demand, lifelong knowledge editing has emerged as a continual approach to modify specific pieces of knowledge without retraining the entire model. Existing parameter editing methods struggle with stability during sequential edits due to catastrophic forgetting. While retrieval-based approaches are proposed to alleviate this issue, their applicability remains limited across various datasets because of high training costs. To address these limitations and enhance scalability in lifelong settings, we propose LightEdit. Our framework first selects relevant knowledge from retrieved information to modify the query effectively. It then incorporates a decoding strategy to suppress the model's original knowledge probabilities, thereby enabling efficient edits based on the selected information. Extensive experiments on ZSRE, Counterfact, and RIPE benchmarks demonstrate that LightEdit outperforms existing lifelong knowledge editing methods. Furthermore, by minimizing training costs, LightEdit achieves cost-effective scalability, enabling easy adaptation to various datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
