强化学习中简化动作搜索的运动基元标注
Annotating Motion Primitives for Simplifying Action Search in Reinforcement Learning
摘要：
大规模环境中的强化学习具有挑战性，因为在特定情况下可以采取许多可能的行动。我们以前已经开发了一种约束的手段，从而加快，通过使用运动原语的搜索过程;运动原语是一系列预先指定的动作采取跨状态系列。作为这项工作的副产品，我们发现，如果运动原语的运动和动作被标记，那么搜索可以进一步加快。由于运动原语最初可能缺乏这样的细节，我们提出了一个理论上的视点不敏感和速度不敏感的方式自动注释的基本运动和动作。我们这样做是通过一个微分几何，时空运动学描述符，它分析了如何在两个运动序列中的实体的姿势随着时间的推移而变化。我们使用这个描述符结合加权最近邻分类器使用一组有限的训练样本来标记原语。在我们的实验中，我们实现了高的运动和动作注释率的人类动作衍生的基元少到一个训练样本。我们还证明，使用准确标记的轨迹的强化学习比标准的强化学习技术更快地产生高性能的策略。这部分是因为运动基元对先前的领域知识进行编码，并在训练期间抢占重新发现该知识的需要。这也是因为智能体可以利用标签来系统地忽略不促进任务目标的动作类，从而减少动作空间。
英文摘要：
Reinforcement learning in large-scale environments is challenging due to the many possible actions that can be taken in specific situations. We have previously developed a means of constraining, and hence speeding up, the search process through the use of motion primitives; motion primitives are sequences of pre-specified actions taken across a state series. As a byproduct of this work, we have found that if the motion primitives' motions and actions are labeled, then the search can be sped up further. Since motion primitives may initially lack such details, we propose a theoretically viewpoint-insensitive and speed-insensitive means of automatically annotating the underlying motions and actions. We do this through a differential-geometric, spatio-temporal kinematics descriptor, which analyzes how the poses of entities in two motion sequences change over time. We use this descriptor in conjunction with a weighted-nearest-neighbor classifier to label the primitives using a limited set of training examples. In our experiments, we achieve high motion and action annotation rates for human-action-derived primitives with as few as one training sample. We also demonstrate that reinforcement learning using accurately labeled trajectories leads to high-performing policies more quickly than standard reinforcement learning techniques. This is partly because motion primitives encode prior domain knowledge and preempt the need to re-discover that knowledge during training. It is also because agents can leverage the labels to systematically ignore action classes that do not facilitate task objectives, thereby reducing the action space.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
