MoEEdit: 高效且路由稳定的知识编辑用于专家混合语言模型
MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMs
AI总结：
MoEEdit提出一种高效且路由稳定的知识编辑方法，用于稀疏混合专家语言模型，通过空空间投影和块坐标下降求解器实现高效率和稳定性。
摘要：
知识编辑（KE）能够对大型语言模型（LLMs）中的事实性内容进行精确修改。现有的KE方法大多针对密集架构设计，限制了它们在日益普及的稀疏混合专家（MoE）模型中的适用性，而这些模型是现代可扩展LLMs的基础。尽管MoEs提供强大的效率和容量扩展能力，但直接适应密集模型编辑器既计算成本高，又容易导致路由分布偏移，从而破坏稳定性和一致性。为了解决这些挑战，我们引入了MoEEdit，这是首个针对MoE LLMs参数修改知识编辑的路由稳定框架。我们的方法通过每个专家的空空间投影重新参数化专家更新，保持路由器输入不变，从而抑制路由偏移。所得到的块结构优化通过块坐标下降（BCD）求解器高效解决。实验表明，MoEEdit在保持高特异性和路由稳定性的同时，实现了最先进的效果和泛化能力，并且具有优越的计算和内存效率。这些结果为稀疏LLMs中的可扩展、精确知识编辑建立了稳健的基础，并突显了路由稳定干预的重要性。
英文摘要：
Knowledge editing (KE) enables precise modifications to factual content in large language models (LLMs). Existing KE methods are largely designed for dense architectures, limiting their applicability to the increasingly prevalent sparse Mixture-of-Experts (MoE) models that underpin modern scalable LLMs. Although MoEs offer strong efficiency and capacity scaling, naively adapting dense-model editors is both computationally costly and prone to routing distribution shifts that undermine stability and consistency. To address these challenges, we introduce MoEEdit, the first routing-stable framework for parameter-modifying knowledge editing in MoE LLMs. Our method reparameterizes expert updates via per-expert null-space projections that keep router inputs invariant and thereby suppress routing shifts. The resulting block-structured optimization is solved efficiently with a block coordinate descent (BCD) solver. Experiments show that MoEEdit attains state-of-the-art efficacy and generalization while preserving high specificity and routing stability, with superior compute and memory efficiency. These results establish a robust foundation for scalable, precise knowledge editing in sparse LLMs and underscore the importance of routing-stable interventions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
