通过世界模型反馈的在线持续强化学习实现自适应机器人智能体
Self-adapting Robotic Agents through Online Continual Reinforcement Learning with World Model Feedback
AI总结：
本文提出一种基于世界模型反馈的在线持续强化学习方法，通过检测分布外事件自动触发微调，实现机器人智能体在运行过程中的自适应能力。
摘要：
作为基于学习的机器人控制器通常是在离线训练并部署固定参数，其在操作过程中应对意外变化的能力有限。受生物启发，本工作提出了一种在线持续强化学习框架，使在部署过程中能够实现自动适应。基于DreamerV3，一种基于模型的强化学习算法，所提出的方法利用世界模型预测残差来检测分布外事件并自动触发微调。通过任务层面性能信号和内部训练指标监控适应进展，允许在没有外部监督和领域知识的情况下评估收敛。该方法在多种现代连续控制问题上得到验证，包括高保真模拟中的四足机器人以及现实世界中的模型车。相关指标及其解释被呈现并讨论，以及由此产生的权衡也被描述。结果展示了如何使自主机器人智能体能够超越静态训练范式，朝着具备自我反思和改进能力的适应系统发展，就像其生物 counterparts 一样。
英文摘要：
As learning-based robotic controllers are typically trained offline and deployed with fixed parameters, their ability to cope with unforeseen changes during operation is limited. Biologically inspired, this work presents a framework for online Continual Reinforcement Learning that enables automated adaptation during deployment. Building on DreamerV3, a model-based Reinforcement Learning algorithm, the proposed method leverages world model prediction residuals to detect out-of-distribution events and automatically trigger finetuning. Adaptation progress is monitored using both task-level performance signals and internal training metrics, allowing convergence to be assessed without external supervision and domain knowledge. The approach is validated on a variety of contemporary continuous control problems, including a quadruped robot in high-fidelity simulation, and a real-world model vehicle. Relevant metrics and their interpretation are presented and discussed, as well as resulting trade-offs described. The results sketch out how autonomous robotic agents could once move beyond static training regimes toward adaptive systems capable of self-reflection and -improvement during operation, just like their biological counterparts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
