基于强化学习的受限三维空间灵巧腿式运动
Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning
AI总结：
针对受限三维空间中腿式机器人导航难题，提出结合经典规划器与强化学习策略的分层运动控制器，在仿真中性能优于现有方法，且成功实现真机部署。
摘要：
利用深度强化学习（RL）的运动控制器近期取得了显著进展，可在崎岖岩石、非刚性地面、湿滑表面等复杂地形上实现快速且鲁棒的运动。然而，现有控制器主要应对机器人下方的地形挑战，针对狭窄隧道、不规则空洞等存在全方位约束的受限三维空间的腿式移动能力研究相对较少。现有基于RL的方法学习的是由速度、身体高度等运动参数表征的参数化运动技能，由此产生的周期性步态模式可能不足以引导机器人穿越复杂的受限三维空间，这类空间既需要灵活的三维避障，又需要鲁棒的腿式运动。为此，我们提出在受限三维空间中从目标导向的导航任务中端到端学习运动技能。为解决追踪远距离导航目标时的效率问题，我们设计了一种分层运动控制器：上层为经典规划器，负责规划航点以抵达远处的全局目标位置；下层为基于RL的策略，经训练后通过生成底层运动指令来跟随这些航点。该方法允许策略在整个解空间中探索自身的运动技能，且能促进局部目标间的平滑过渡，从而实现朝向远距离目标的长期导航。仿真实验中，我们的分层方法成功穿越了复杂的受限三维环境，性能优于纯端到端学习方法和参数化运动技能方法。我们进一步验证了该仿真训练的控制器可成功部署到真实机器人上。
英文摘要：
Recent advances of locomotion controllers utilizing deep reinforcement learning (RL) have yielded impressive results in terms of achieving rapid and robust locomotion across challenging terrain, such as rugged rocks, non-rigid ground, and slippery surfaces. However, while these controllers primarily address challenges underneath the robot, relatively little research has investigated legged mobility through confined 3D spaces, such as narrow tunnels or irregular voids, which impose all-around constraints. The cyclic gait patterns resulted from existing RL-based methods to learn parameterized locomotion skills characterized by motion parameters, such as velocity and body height, may not be adequate to navigate robots through challenging confined 3D spaces, requiring both agile 3D obstacle avoidance and robust legged locomotion. Instead, we propose to learn locomotion skills end-to-end from goal-oriented navigation in confined 3D spaces. To address the inefficiency of tracking distant navigation goals, we introduce a hierarchical locomotion controller that combines a classical planner tasked with planning waypoints to reach a faraway global goal location, and an RL-based policy trained to follow these waypoints by generating low-level motion commands. This approach allows the policy to explore its own locomotion skills within the entire solution space and facilitates smooth transitions between local goals, enabling long-term navigation towards distant goals. In simulation, our hierarchical approach succeeds at navigating through demanding confined 3D environments, outperforming both pure end-to-end learning approaches and parameterized locomotion skills. We further demonstrate the successful real-world deployment of our simulation-trained controller on a real robot.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
