arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.02471cs.ROcs.LGcs.SYeess.SY
收藏

配置路径控制

Configuration Path Control

Harik Shazeer Labs(哈里克·沙齐尔实验室)

Sergey Pankov

摘要:

强化学习方法通常会产生脆弱的策略--在训练过程中表现良好的策略,但在直接训练经验之外的推广效果很差,因此在小干扰下变得不稳定。为了解决这个问题,我们提出了一种方法,用于稳定的控制策略的配置路径的空间。它在训练后应用,完全依赖于训练期间产生的数据以及瞬时控制矩阵估计。经验上的平面双足步行机受到各种扰动的方法进行评估。通过强化学习获得的控制策略进行比较,其稳定的同行。在不同的实验中,我们发现两到四倍的稳定性增加,当测量的扰动幅度。我们还提供了一个零动态解释我们的方法。

英文摘要:

Reinforcement learning methods often produce brittle policies -- policies that perform well during training, but generalize poorly beyond their direct training experience, thus becoming unstable under small disturbances. To address this issue, we propose a method for stabilizing a control policy in the space of configuration paths. It is applied post-training and relies purely on the data produced during training, as well as on an instantaneous control-matrix estimation. The approach is evaluated empirically on a planar bipedal walker subjected to a variety of perturbations. The control policies obtained via reinforcement learning are compared against their stabilized counterparts. Across different experiments, we find two- to four-fold increase in stability, when measured in terms of the perturbation amplitudes. We also provide a zero-dynamics interpretation of our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑