CLaaS: 作为服务的持续学习，用于样本高效的在线学习
CLaaS: Continual learning as a service for sample efficient online learning
AI总结：
提出CLaaS系统，通过经验回放缓冲区实现异步训练中的梯度复用，在对抗性任务中展示参数更新优于上下文学习的前向迁移和遗忘减少。
摘要：
部署的大型语言模型代理必须适应动态环境中的分布偏移。理想情况下，可以从累积的代理经验中进行适应，并在转移到未来任务时保留先前的能力。然而，由于真实环境无法轻易重置，每个场景中代理的动作和环境转换只能采样一次。为此，我们研究了一种体验式和在线持续学习设置，其中代理从一系列场景中学习。我们提出了持续学习即服务（CLaaS），这是一个系统，使代理能够在部署期间改进，并通过聊天API抽象化。为了提高样本效率，CLaaS将轨迹存储在经验回放缓冲区中，以便在异步训练期间重用梯度。我们在对抗性任务上评估了CLaaS，证明参数更新比上下文学习具有更好的前向迁移和更少的遗忘，其中回放是样本效率的关键选择。
英文摘要：
Deployed large language model agents must adapt to distribution shift in dynamic environments. Ideally, adaptation can be performed from accumulated agent experiences and retain prior capabilities while transferring to future tasks. However, agent actions and environmental transitions can only be sampled once per scenario, as real-world environments cannot be trivially reset. To this end, we investigate an experiential and online continual learning setting in which agents learn from a stream of scenarios. We propose continual learning as-a-service (CLaaS), a system which enables agents to improve during deployment, abstracted behind a chat API. To increase sample efficiency, CLaaS stores rollouts in an experience replay buffer for gradient reuse during asynchronous training. We evaluate CLaaS on an adversarial task, demonstrating that parametric updates lead to superior forward transfer and less forgetting than in-context learning, with replay being a critical choice for sample efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
