MEMoE：利用混合专家适配器增强模型编辑
MEMoE: Enhancing Model Editing with Mixture of Experts Adaptors
AI总结：
针对模型编辑中泛化性与局部性难以平衡的问题，提出MEMoE适配器，利用混合专家架构和知识锚点路由策略，在不改变原始参数的前提下更新知识，在批量编辑任务中实现了卓越的整体性能与平衡。
摘要：
模型编辑旨在高效改变大型语言模型（LLMs）在特定范围内的行为，同时确保对其他输入无负面影响。近年来出现了多种模型编辑方法，但这些方法要么整体性能不佳，要么难以在泛化性与局部性之间取得平衡。我们提出了MEMoE，这是一种利用混合专家架构与知识锚点路由策略的模型编辑适配器。MEMoE通过旁路MoE结构更新知识，保持原始参数不变以保留LLMs的通用能力。此外，知识锚点路由确保需要相似知识的输入被路由至同一专家，从而增强更新知识的泛化性。实验结果表明，我们的方法在批量编辑和顺序批量编辑任务上均表现出优越性，展现出卓越的整体性能以及泛化性与局部性之间的出色平衡。我们的代码将会发布。
英文摘要：
Model editing aims to efficiently alter the behavior of Large Language Models (LLMs) within a desired scope, while ensuring no adverse impact on other inputs. Recent years have witnessed various model editing methods been proposed. However, these methods either exhibit poor overall performance or struggle to strike a balance between generalization and locality. We propose MEMoE, a model editing adapter utilizing a Mixture of Experts (MoE) architecture with a knowledge anchor routing strategy. MEMoE updates knowledge using a bypass MoE structure, keeping the original parameters unchanged to preserve the general ability of LLMs. And, the knowledge anchor routing ensures that inputs requiring similar knowledge are routed to the same expert, thereby enhancing the generalization of the updated knowledge. Experimental results show the superiority of our approach over both batch editing and sequential batch editing tasks, exhibiting exceptional overall performance alongside outstanding balance between generalization and locality. Our code will be available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
