arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.07599cs.ROcs.AI
收藏

Meta-强化学习在机器人动态和环境变化中的自适应电机控制

Meta-Reinforcement Learning for Adaptive Motor Control in Changing Robot Dynamics and Environments

ENS Rennes(雷恩高等师范学院);the University of Edinburgh(爱丁堡大学)

Timothée Anne, Jack Wilkinson, Zhibin Li

摘要:

这项工作开发了一种元学习方法,可以根据不同的变化条件动态调整控制策略,以实现鲁棒运动。该方法不断更新交互模型,对估计的状态-动作轨迹的可行动作序列进行采样,然后应用最优动作来最大化奖励。为了实现在线模型自适应,我们提出的方法学习每个训练条件的不同潜向量,这些潜向量是根据新收集的数据在线选择的。我们的工作设计了适当的状态空间和奖励函数,并优化可行的行动在MPC的方式,然后直接在关节空间采样考虑约束,因此不需要事先设计的具体步行步态。我们进一步证明了机器人的能力,检测意外的变化,在互动和适应控制策略迅速。在物理模拟中对SpotMicro机器人进行的广泛验证显示了在不同的地面摩擦力,外部推力和不同的机器人模型(包括硬件故障和变化)下的自适应和强大的运动技能。

英文摘要:

This work developed a meta-learning approach that adapts the control policy on the fly to different changing conditions for robust locomotion. The proposed method constantly updates the interaction model, samples feasible sequences of actions of estimated the state-action trajectories, and then applies the optimal actions to maximize the reward. To achieve online model adaptation, our proposed method learns different latent vectors of each training condition, which are selected online given the newly collected data. Our work designs appropriate state space and reward functions, and optimizes feasible actions in an MPC fashion which are then sampled directly in the joint space considering constraints, hence requiring no prior design of specific walking gaits. We further demonstrate the robot's capability of detecting unexpected changes during interaction and adapting control policies quickly. The extensive validation on the SpotMicro robot in a physics simulation shows adaptive and robust locomotion skills under varying ground friction, external pushes, and different robot models including hardware faults and changes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑