通过定制化专家网络实现可扩展的模型编辑
Scalable Model Editing via Customized Expert Networks
AI总结：
本文提出SCEN方法，通过为每条知识训练轻量级专家网络及索引神经元的两阶段连续训练范式，在ZsRE和Hallucination基准上基于Llama2实现了优于主流方法的可扩展模型编辑效果。
摘要：
解决大语言模型中的幻觉和知识过时问题对其可靠应用至关重要。模型编辑为以低成本方式缓解这些挑战提供了一条有前景的途径。然而，现有方法往往存在泛化效果不理想以及对未编辑样本产生非预期影响的问题。为克服这些局限，我们提出了一种新方法：通过定制化专家网络实现可扩展的模型编辑（SCEN），这是一种两阶段连续训练范式。具体而言，在第一阶段，我们为每条需要更新的知识分别训练轻量级专家网络。随后，我们为每个专家训练一个相应的索引神经元，以控制该专家的激活状态。我们通过微调先进的开源大语言模型 Llama2，在 ZsRE 和 Hallucination 基准上开展了一系列实验，与当前主流方法相比取得了最先进的结果。我们的代码可在 https://github.com/TAL-auroraX/SCEN 获取。
英文摘要：
Addressing the issues of hallucinations and outdated knowledge in large language models is critical for their reliable application. Model Editing presents a promising avenue for mitigating these challenges in a cost-effective manner. However, existing methods often suffer from unsatisfactory generalization and unintended effects on non-edited samples. To overcome these limitations, we introduce a novel approach: Scalable Model Editing via Customized Expert Networks (SCEN), which is a two-stage continuous training paradigm. Specifically, in the first stage, we train lightweight expert networks individually for each piece of knowledge that needs to be updated. Subsequently, we train a corresponding indexing neuron for each expert to control the activation state of that expert. We conducted a series of experiments on the ZsRE and Hallucination benchmarks by tuning the advanced open-source LLM, Llama2, achieving state-of-the-art results compared to current mainstream methods. Our code is available at https://github.com/TAL-auroraX/SCEN.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
