带有预算约束的本地语言模型与云卸载决策的联合持续学习
Joint Continual Learning of Local Language Models and Cloud Offloading Decisions with Budget Constraints
AI总结：
本文提出DA-GRPO方法，通过联合学习本地语言模型和云卸载决策，有效解决持续学习中的预算约束问题，提升任务准确性并减少遗忘。
摘要：
本地部署的小语言模型（SLMs）必须在严格的内存和计算限制下持续支持多样化任务，使选择依赖云上的大语言模型（LLMs）成为不可避免。在持续学习过程中调节云辅助存在挑战，因为朴素的基于奖励的强化学习往往导致不稳定的卸载行为，并加剧任务分布变化时的灾难性遗忘。我们提出了DA-GRPO，这是Group Relative Policy Optimization的双优势扩展，它将云使用约束直接纳入优势计算中，避免了固定奖励塑造和外部路由模型。这种设计使本地模型能够共同学习任务能力和协作行为，允许在训练后自然产生云请求，同时遵守规定的帮助预算。在数学推理和代码生成基准测试中，DA-GRPO在切换后准确性有所提高，显著减少了遗忘，并在云使用方面保持稳定，相比以往的协作和路由方法有所改进。
英文摘要：
Locally deployed Small Language Models (SLMs) must continually support diverse tasks under strict memory and computation constraints, making selective reliance on cloud Large Language Models (LLMs) unavoidable. Regulating cloud assistance during continual learning is challenging, as naive reward-based reinforcement learning often yields unstable offloading behavior and exacerbates catastrophic forgetting as task distributions shift. We propose DA-GRPO, a dual-advantage extension of Group Relative Policy Optimization that incorporates cloud-usage constraints directly into advantage computation, avoiding fixed reward shaping and external routing models. This design enables the local model to jointly learn task competence and collaboration behavior, allowing cloud requests to emerge naturally during post-training while respecting a prescribed assistance budget. Experiments on mathematical reasoning and code generation benchmarks show that DA-GRPO improves post-switch accuracy, substantially reduces forgetting, and maintains stable cloud usage compared to prior collaborative and routing-based approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
