通过持续指令微调实现自进化大语言模型
Self-Evolving LLMs via Continual Instruction Tuning
AI总结：
提出MoE-CL，一种参数高效对抗性混合专家框架，通过双LoRA专家和任务感知判别器实现自进化持续指令微调，缓解灾难性遗忘并提升跨任务泛化，在工业基准和A/B测试中显著降低人工审核成本。
摘要：
在现实工业环境中，大语言模型（LLMs）必须持续学习以跟上多样且不断演变的任务，这要求模型在动态数据分布下通过自进化来精炼知识。然而，现有的持续学习（CL）方法，如回放和参数隔离，常常遭受灾难性遗忘：在新任务上训练会因过度拟合新分布并削弱泛化能力而降低在早期任务上的性能。我们提出MoE-CL，一种面向工业规模、自进化持续指令微调大语言模型的参数高效对抗性混合专家框架。MoE-CL采用双专家设计：（1）每个任务一个专用的LoRA专家，通过参数独立性保留任务特定知识，缓解遗忘；（2）一个共享LoRA专家以实现跨任务迁移。为防止通过共享路径传递与任务无关的噪声，我们在生成对抗网络（GAN）中集成一个任务感知判别器。该判别器鼓励共享专家在顺序训练期间仅传递与任务对齐的信息。通过对抗学习，共享专家获得模仿判别器的通用表示，而专用专家保留任务特定细节，平衡知识保留与跨任务泛化，从而支持自进化。在公开的MTL5基准和工业Tencent3基准上的大量实验验证了MoE-CL在持续指令微调中的有效性。在腾讯视频平台内容合规审查的真实A/B测试中，MoE-CL将人工审核成本降低了15.3%。这些结果表明，MoE-CL对于持续适应和稳定迁移至关重要的工业级大规模部署具有实用性。
英文摘要：
In real-world industrial settings, large language models (LLMs) must learn continually to keep pace with diverse and evolving tasks, requiring self-evolution to refine knowledge under dynamic data distributions. However, existing continual learning (CL) approaches, such as replay and parameter isolation, often suffer from catastrophic forgetting: training on new tasks degrades performance on earlier ones by overfitting to the new distribution and weakening generalization.We propose MoE-CL, a parameter-efficient adversarial mixture-of-experts framework for industrial-scale, self-evolving continual instruction tuning of LLMs. MoE-CL uses a dual-expert design: (1) a dedicated LoRA expert per task to preserve task-specific knowledge via parameter independence, mitigating forgetting; and (2) a shared LoRA expert to enable cross-task transfer. To prevent transferring task-irrelevant noise through the shared pathway, we integrate a task-aware discriminator within a GAN. The discriminator encourages the shared expert to pass only task-aligned information during sequential training. Through adversarial learning, the shared expert acquires generalized representations that mimic the discriminator, while dedicated experts retain task-specific details, balancing knowledge retention and cross-task generalization and thereby supporting self-evolution.Extensive experiments on the public MTL5 benchmark and an industrial Tencent3 benchmark validate the effectiveness of MoE-CL for continual instruction tuning. In real-world A/B testing for content compliance review on the Tencent Video platform, MoE-CL reduced manual review costs by 15.3%. These results demonstrate that MoE-CL is practical for large-scale industrial deployment where continual adaptation and stable transfer are critical.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
