arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.02249cs.LGcs.SYeess.SY
收藏

基于$\mathcal{L}_1$自适应控制的强化学习策略鲁棒化

Robustifying Reinforcement Learning Policies with $\mathcal{L}_1$ Adaptive Control

Yikun Cheng, Pan Zhao, Manan Gandhi, Bo Li, Evangelos Theodorou, Naira Hovakimyan

摘要:

由于动态变化的存在,在标称环境中训练的强化学习(RL)策略可能在新的/扰动的环境中失败。现有的鲁棒方法试图通过鲁棒或对抗训练来获得针对所有设想的动态变化场景的固定策略。这些方法可能会导致保守的性能,由于强调最坏的情况下,往往涉及繁琐的修改训练环境。我们提出了一种方法来robustifying一个预先训练的非鲁棒RL政策与$\mathcal{L}_1$自适应控制。利用$\mathcal{L}_1$控制律在动态变化的快速估计和主动补偿中的能力,我们的方法可以显着提高在标准中训练的RL策略的鲁棒性(即,非鲁棒的)方式,无论是在模拟器中还是在现实世界中。数值实验验证了该方法的有效性。

英文摘要:

A reinforcement learning (RL) policy trained in a nominal environment could fail in a new/perturbed environment due to the existence of dynamic variations. Existing robust methods try to obtain a fixed policy for all envisioned dynamic variation scenarios through robust or adversarial training. These methods could lead to conservative performance due to emphasis on the worst case, and often involve tedious modifications to the training environment. We propose an approach to robustifying a pre-trained non-robust RL policy with $\mathcal{L}_1$ adaptive control. Leveraging the capability of an $\mathcal{L}_1$ control law in the fast estimation of and active compensation for dynamic variations, our approach can significantly improve the robustness of an RL policy trained in a standard (i.e., non-robust) way, either in a simulator or in the real world. Numerical experiments are provided to validate the efficacy of the proposed approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑