OGMP：面向敏捷与多才多艺机器人控制的神谕引导多模式策略
OGMP: Oracle Guided Multi-mode Policies for Agile and Versatile Robot Control
AI总结：
本文提出神谕引导的多模式策略优化方法，通过定义神谕先验和任务关键模式，使16自由度双足机器人HECTOR在模拟和现实中实现跑酷与跳水等敏捷控制，并成功迁移至真实机器人。
摘要：
强化学习在机器人控制中的有效性依赖于针对特定任务的先验知识和启发式方法的定制整合，以进行有效探索，这对其直接应用于复杂任务构成挑战，并需要一种统一的方法。在这项工作中，我们定义了一类通用的先验知识，称为神谕（oracles），在训练过程中以闭环方式查询时，它们会生成状态参考。通过将允许的状态限制在神谕的假设解周围，我们提出了一种与任务无关的神谕引导策略优化方法。为了增强模块化，我们引入了任务关键模式（task-vital modes），表明掌握一组紧凑的模式及其转换的策略能够处理无限时域的任务。例如，要在无限长的跑道上进行跑酷，策略必须学会跳跃、跨越、步态调整以及在这些模式之间有效转换。我们在具有挑战性的双足控制任务中验证了该方法：使用16自由度动态双足机器人HECTOR进行跑酷和跳水。我们的方法为每个任务产生单一策略，能够在模拟中解决跨多种跑道的跑酷以及从高达2米的不同高度进行的全向跳水，展示了多样的敏捷性。我们成功展示了跑酷的仿真到现实迁移，包括跨越高达腿长105%的间隙、跳跃高达机器人标称高度20%的障碍物，以及以高达0.6米/秒的速度进行步态调整，并在真实机器人上实现了这些模式之间的有效转换。
英文摘要：
The efficacy of reinforcement learning for robot control relies on the tailored integration of task-specific priors and heuristics for effective exploration, which challenges their straightforward application to complex tasks and necessitates a unified approach. In this work, we define a general class for priors called oracles that generate state references when queried in a closed-loop manner during training. By bounding the permissible state around the oracle's ansatz, we propose a task-agnostic oracle-guided policy optimization. To enhance modularity, we introduce task-vital modes, showing that a policy mastering a compact set of modes and transitions can handle infinite-horizon tasks. For instance, to perform parkour on an infinitely long track, the policy must learn to jump, leap, pace, and transition between these modes effectively. We validate this approach in challenging bipedal control tasks: parkour and diving using a 16 DoF dynamic bipedal robot, HECTOR. Our method results in a single policy per task, solving parkour across diverse tracks and omnidirectional diving from varied heights up to 2m in simulation, showcasing versatile agility. We demonstrate successful sim-to-real transfer of parkour, including leaping over gaps up to 105 % of the leg length, jumping over blocks up to 20 % of the robot's nominal height, and pacing at speeds of up to 0.6 m/s, along with effective transitions between these modes in the real robot.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
