arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.27353cs.RO
收藏

通过变分神经动力学的持续机器人策略学习

Continual Robot Policy Learning via Variational Neural Dynamics

Robotics and Perception Group, University of Zurich(苏黎世大学机器人感知组)

Jiaxu Xing, Zhiyuan Zhu, Yunfan Ren, Ismail Geles, Yifan Zhai, Rudolf Reiter, Davide Scaramuzza

AI总结:

提出持续学习框架,结合物理先验与神经残差建模隐藏动态,通过可微仿真和在线条件推断提升策略在未知干扰下的性能。

摘要:

部署在现实世界中的机器人很少在单一固定动力学模型下运行:风向变化、负载变化、电池耗尽、接触变化以及硬件磨损。然而,大多数基于学习的控制器仅训练一次并部署,仿佛学习已经完成。这阻止了机器人利用部署经验进一步提高任务性能。在这项工作中,我们提出了一个持续学习框架,利用现实世界经验在隐藏和重复出现的动力学下改进机器人策略。我们的方法通过将分析物理先验与用于未建模效应的神经残差相结合,从真实状态-动作轨迹中学习条件感知的动力学模型。一个循环编码器从最近的交互中推断当前的隐藏条件,该估计同时调节残差模型和策略。策略学习通过可微仿真进行,使用从潜在模型中采样的多样化学习动力学。在部署时,这些采样条件被从最近的真实交互中在线推断的条件所替代,使得策略能够通过识别而非残差重新拟合来恢复重复出现的动力学。通过广泛的仿真研究和真实世界实验,我们证明了该框架在多种未观测干扰下提高了策略性能。在变化的强风中进行的真实四旋翼轨迹跟踪实验中,策略在大约1秒内从重复出现的干扰中恢复,比在线残差重新拟合快约5倍。与最先进的在线自适应方法相比,它还将大干扰悬停和跟踪误差分别减少了65.7%和53.3%。

英文摘要:

Robots deployed in the real world rarely operate under a single fixed dynamics model: wind changes, payloads vary, batteries drain, contacts shift, and hardware wears. Yet most learning-based controllers are trained once and deployed as if learning were complete. This prevents the robot from using deployment experience to further improve task performance. In this work, we propose a continual learning framework that uses real-world experience to improve robot policies under hidden and recurring dynamics. Our method learns a condition-aware dynamics model from real state-action trajectories by combining an analytical physics prior with a neural residual for unmodeled effects. A recurrent encoder infers the current hidden condition from recent interaction, and this estimate conditions both the residual model and the policy. Policy learning is performed via differentiable simulation using diverse learned dynamics sampled from the latent model. At deployment, these sampled conditions are replaced by conditions inferred online from recent real interaction, allowing the policy to recover recurring dynamics by recognition rather than residual re-fitting. Through extensive simulation studies and real-world experiments, we demonstrate that the framework improves policy performance under diverse unobserved disturbances. On real quadrotor trajectory tracking under changing wind, the policy recovers from recurring disturbances in roughly 1s, about 5x faster than online residual re-fitting. It also reduces large-disturbance hover and tracking errors by 65.7% and 53.3% over the state-of-the-art online adaptation approaches

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑