arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 1708.02205cs.RO
收藏

基于强化学习和新型全身控制器的鲁棒动态运动

Robust Dynamic Locomotion via Reinforcement Learning and Novel Whole Body Controller

University of Texas at Austin(德克萨斯大学奥斯汀分校)

Donghyun Kim, Jaemin Lee, Luis Sentis

摘要:

我们提出了一种鲁棒动态步行控制器,由动态运动规划器、鲁棒性的强化学习过程和新型全身运动控制器(WBLC)组成。以前的方法指定的位置或时间的步骤,然而,建议的运动规划同时计算这两个参数作为运动输出。我们的运动策略依赖于设计一个强化学习(RL)方法来实现健壮的行走。学习的策略生成多步行走模式,并且该过程足够快以适合于实时控制。对于学习,我们设计了一种RL策略,该策略使用相空间规划器(PSP)和线性倒立摆模型来使问题易于处理并且非常快速。然后,学习的策略用于向WBLC提供基于目标的命令,WBLC计算要在全人形机器人中执行的扭矩命令。WBLC结合了多个优先级任务,并根据实际的不等式约束计算相关的反作用力。新的配方包括各种雅可比矩阵的时间导数的有效计算。这提供了对快速运动的高保真动态控制。更具体地说,我们分别利用李群算子和操作空间动力学计算各种任务的雅可比矩阵的时间导数和质心动量任务的雅可比矩阵。RL-PSP和WBLC的集成提供了高度鲁棒、通用和实用的运动,包括行走时的转向和在0.1秒的间隔内处理高达520 N的推力干扰。理论和数值结果进行了测试,通过三维物理为基础的仿真的人形机器人瓦尔基里。

英文摘要:

We propose a robust dynamic walking controller consisting of a dynamic locomotion planner, a reinforcement learning process for robustness, and a novel whole-body locomotion controller (WBLC). Previous approaches specify either the position or the timing of steps, however, the proposed locomotion planner simultaneously computes both of these parameters as locomotion outputs. Our locomotion strategy relies on devising a reinforcement learning (RL) approach for robust walking. The learned policy generates multi step walking patterns, and the process is quick enough to be suitable for real-time controls. For learning, we devise an RL strategy that uses a phase space planner (PSP) and a linear inverted pendulum model to make the problem tractable and very fast. Then, the learned policy is used to provide goal-based commands to the WBLC, which calculates the torque commands to be executed in full-humanoid robots. The WBLC combines multiple prioritized tasks and calculates the associated reaction forces based on practical inequality constraints. The novel formulation includes efficient calculation of the time derivatives of various Jacobians. This provides high-fidelity dynamic control of fast motions. More specifically, we compute the time derivative of the Jacobian for various tasks and the Jacobian of the centroidal momentum task by utilizing Lie group operators and operational space dynamics respectively. The integration of RL-PSP and the WBLC provides highly robust, versatile, and practical locomotion including steering while walking and handling push disturbances of up to 520 N during an interval of 0.1 sec. Theoretical and numerical results are tested through a 3D physics-based simulation of the humanoid robot Valkyrie.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑