面向动态双足移动操作的多模式偏好引导预言机策略
Preferenced Oracle Guided Multi-mode Policies for Dynamic Bipedal Loco-Manipulation
AI总结：
提出偏好引导预言机多模式策略（OGMP），通过混合自动机预言机和偏好奖励，学习单一策略实现动态双足移动操作，在足球和搬箱任务中验证了跨形态的全身控制能力。
摘要：
动态移动操作要求有效的全身控制以及与物体和环境的丰富接触交互。现有的基于学习的控制综合依赖于训练低级技能策略，并通过高级策略或手工设计的有限状态机进行显式切换，导致准静态行为。相比之下，足球等动态任务要求机器人跑向球、减速至最佳接近方式以运球，并最终射门得分——这是一个平滑运动的连续体。为此，我们提出偏好引导预言机多模式策略（OGMP），以学习一个单一策略，掌握解决单物体移动操作任务所需的全部模式及优选的转换序列。我们设计混合自动机作为预言机，生成具有连续动态和离散模式跳变的参考，通过有界探索进行引导式策略优化。为了强制学习期望的模式转换序列，我们提出了一种与任务无关的偏好奖励，以增强性能。所提出的方法在足球和全方位移动箱子等任务中通过全身控制展示了成功的移动操作。在足球中，单一策略学会最优地到达球、过渡到接触丰富的运球，并执行成功的射门和停球。利用预言机的抽象，我们在不同形态的机器人上解决每个移动操作任务，包括HECTOR V1、Berkeley Humanoid、Unitree G1和H1，使用相同的奖励定义和权重。
英文摘要：
Dynamic loco-manipulation calls for effective whole-body control and contact-rich interactions with the object and the environment. Existing learning-based control synthesis relies on training low-level skill policies and explicitly switching with a high-level policy or a hand-designed finite state machine, leading to quasi-static behaviors. In contrast, dynamic tasks such as soccer require the robot to run towards the ball, decelerate to an optimal approach to dribble, and eventually kick a goal - a continuum of smooth motion. To this end, we propose Preferenced Oracle Guided Multi-mode Policies (OGMP) to learn a single policy mastering all the required modes and preferred sequence of transitions to solve uni-object loco-manipulation tasks. We design hybrid automatons as oracles to generate references with continuous dynamics and discrete mode jumps to perform a guided policy optimization through bounded exploration. To enforce learning a desired sequence of mode transitions, we present a task-agnostic preference reward that enhances performance. The proposed approach demonstrates successful loco-manipulation for tasks like soccer and moving boxes omnidirectionally through whole-body control. In soccer, a single policy learns to optimally reach the ball, transition to contact-rich dribbling, and execute successful goal kicks and ball stops. Leveraging the oracle's abstraction, we solve each loco-manipulation task on robots with varying morphologies, including HECTOR V1, Berkeley Humanoid, Unitree G1, and H1, using the same reward definition and weights.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
