基于语义路由的LoRA的可逆终身模型编辑
Reversible Lifelong Model Editing via Semantic Routing-Based LoRA
AI总结：
本文提出SoLA框架，通过语义路由实现可逆的终身模型编辑，解决持续更新导致的语义漂移和知识遗忘问题，支持精确撤销特定编辑。
摘要：
现实世界的动态演变要求大语言模型具备模型编辑能力。尽管现有方法探索模块隔离或参数高效策略，但仍因持续更新导致语义漂移或知识遗忘。为此，我们提出SoLA，一种基于语义路由的LoRA框架用于终身模型编辑。在SoLA中，每个编辑被封装为独立的LoRA模块，在训练后冻结并通过语义路由映射到输入，允许通过语义匹配动态激活LoRA模块。此机制避免了集群更新导致的语义漂移，缓解了参数共享带来的灾难性遗忘。更重要的是，SoLA通过移除关键项从语义路由中实现对特定编辑的精确撤销，恢复模型的原始行为。据我们所知，这种可逆回滚编辑能力是现有文献中首次实现的。此外，SoLA将决策过程整合到编辑层中，消除了对辅助路由网络的需要，实现了端到端的决策过程。大量实验表明，SoLA有效学习并保留编辑知识，实现了准确、高效且可逆的终身模型编辑。
英文摘要：
The dynamic evolution of real-world necessitates model editing within Large Language Models. While existing methods explore modular isolation or parameter-efficient strategies, they still suffer from semantic drift or knowledge forgetting due to continual updating. To address these challenges, we propose SoLA, a Semantic routing-based LoRA framework for lifelong model editing. In SoLA, each edit is encapsulated as an independent LoRA module, which is frozen after training and mapped to input by semantic routing, allowing dynamic activation of LoRA modules via semantic matching. This mechanism avoids semantic drift caused by cluster updating and mitigates catastrophic forgetting from parameter sharing. More importantly, SoLA supports precise revocation of specific edits by removing key from semantic routing, which restores model's original behavior. To our knowledge, this reversible rollback editing capability is the first to be achieved in existing literature. Furthermore, SoLA integrates decision-making process into edited layer, eliminating the need for auxiliary routing networks and enabling end-to-end decision-making process. Extensive experiments demonstrate that SoLA effectively learns and retains edited knowledge, achieving accurate, efficient, and reversible lifelong model editing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
