Split-on-Share：任务无关持续学习的稀疏专家混合
Split-on-Share: Mixture of Sparse Experts for Task-Agnostic Continual Learning
AI总结：
SETA通过分解模型为模块化子空间解决持续学习中的可塑性-稳定性矛盾，通过弹性权重锚定保护共享知识并自动检索任务专用专家组合，优于现有参数高效微调方法。
摘要：
在大型语言模型中，持续学习受到可塑性-稳定性困境的阻碍，即获得新能力往往导致先前知识的灾难性遗忘。现有方法通常对参数统一处理，未能区分特定任务知识和共享能力。我们提出SETA框架，通过将模型分解为模块化子空间解决该矛盾。不同于标准更新中任务竞争同一参数，SETA将知识分为独特专家（隔离任务特定模式）和共享专家（捕获共同特征）。该结构通过弹性权重锚定维持，使统一门控网络在推理时自动检索每个任务的正确专家组合。在多样化的领域特定和通用基准测试中，SETA consistently 超过 state-of-the-art 参数高效微调连续学习方法。
英文摘要：
Continual learning in Large Language Models (LLMs) is hindered by the plasticity-stability dilemma, where acquiring new capabilities often leads to catastrophic forgetting of previous knowledge. Existing methods typically treat parameters uniformly, failing to distinguish between specific task knowledge and shared capabilities. We introduce Mixture of Sparse Experts for Task-Agnostic Continual Learning, referred to as SETA, a framework that resolves the plasticity-stability conflict by decomposing the model into modular subspaces. Unlike standard updates, where tasks compete for the same parameters, SETA separates knowledge into unique experts, designed to isolate task-specific patterns, and shared experts, responsible for capturing common features. This structure is maintained through elastic weight anchoring, which protects critical shared knowledge and enables a unified gating network to automatically retrieve the correct expert combination for each task during inference. Extensive experiments across diverse domain-specific and general benchmarks demonstrate that SETA consistently outperforms state-of-the-art parameter-efficient fine-tuning-based continual learning methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
