几何织物：策略学习的安全引导介质
Geometric Fabrics: a Safe Guiding Medium for Policy Learning
AI总结：
本文提出几何织物框架，通过人工二阶动力学形成行为动力学，为强化学习策略提供安全引导的动作空间，并在高自由度机器人手立方体手中重定向任务中验证其有效性。
摘要：
机器人策略始终受制于复杂的二阶动力学，这些动力学将其动作与最终状态纠缠在一起。在强化学习（RL）情境中，策略必须通过大量经验和复杂奖励函数来解读这些复杂交互，才能学会如何完成任务。此外，策略通常直接向控制器（如操作空间控制（OSC）或关节PD控制）发出动作，这会在任务空间或关节空间中产生朝向这些动作目标的直线运动。然而，这些空间中的直线运动在很大程度上无法捕捉机器人需要展现的丰富非线性行为，从而将发现这些行为的负担更完全地转移给了智能体。与这些较简单的控制器不同，几何织物通过基于非线性几何的人工二阶动力学，捕捉到一组更丰富且更理想的行为。这些人工动力学通过适当的控制律改变机器人的非受控动力学，从而形成行为动力学。行为动力学解锁了新的动作空间以及安全、引导性的行为，强化学习策略可在其上训练。行为动力学使类似bang-bang的强化学习策略动作仍对真实机器人安全，简化了奖励工程，并有助于编排真实世界的高性能策略。我们更一般地描述了该框架，并针对高自由度机器人手对立方体进行灵巧手中重定向的问题创建了一个具体实例。
英文摘要：
Robotics policies are always subjected to complex, second order dynamics that entangle their actions with resulting states. In reinforcement learning (RL) contexts, policies have the burden of deciphering these complicated interactions over massive amounts of experience and complex reward functions to learn how to accomplish tasks. Moreover, policies typically issue actions directly to controllers like Operational Space Control (OSC) or joint PD control, which induces straightline motion towards these action targets in task or joint space. However, straightline motion in these spaces for the most part do not capture the rich, nonlinear behavior our robots need to exhibit, shifting the burden of discovering these behaviors more completely to the agent. Unlike these simpler controllers, geometric fabrics capture a much richer and desirable set of behaviors via artificial, second order dynamics grounded in nonlinear geometry. These artificial dynamics shift the uncontrolled dynamics of a robot via an appropriate control law to form behavioral dynamics. Behavioral dynamics unlock a new action space and safe, guiding behavior over which RL policies are trained. Behavioral dynamics enable bang-bang-like RL policy actions that are still safe for real robots, simplify reward engineering, and help sequence real-world, high-performance policies. We describe the framework more generally and create a specific instantiation for the problem of dexterous, in-hand reorientation of a cube by a highly actuated robot hand.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
