arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.07013cs.LGcs.AIcs.RO
收藏

多不确定集上的鲁棒策略学习

Robust Policy Learning over Multiple Uncertainty Sets

Stanford University(斯坦福大学);Facebook AI Research(脸书人工智能研究院)

Annie Xie, Shagun Sodhani, Chelsea Finn, Joelle Pineau, Amy Zhang

摘要:

强化学习(RL)代理需要对安全关键环境中的变化具有鲁棒性。虽然系统辨识方法提供了一种从在线体验中推断变化的方法,但它们在无法快速识别的环境中可能会失败。另一种主要的方法是鲁棒强化学习,它产生了一个可以处理最坏情况的策略,但这些方法通常被设计为实现对必须在训练时指定的单个不确定性集的鲁棒性。为了更一般的解决方案,我们制定了多集鲁棒性问题,以学习对不同扰动集具有鲁棒性的策略。然后,我们设计了一个算法,享受系统辨识和鲁棒RL的好处:它减少了不确定性,在可能的情况下,给出了一些相互作用,但仍然可以采取行动,相对于其余的不确定性鲁棒。在一组不同的控制任务,我们的方法证明了改进的最坏情况下的性能相比,以前的方法基于系统辨识和鲁棒RL单独在新的环境。

英文摘要:

Reinforcement learning (RL) agents need to be robust to variations in safety-critical environments. While system identification methods provide a way to infer the variation from online experience, they can fail in settings where fast identification is not possible. Another dominant approach is robust RL which produces a policy that can handle worst-case scenarios, but these methods are generally designed to achieve robustness to a single uncertainty set that must be specified at train time. Towards a more general solution, we formulate the multi-set robustness problem to learn a policy robust to different perturbation sets. We then design an algorithm that enjoys the benefits of both system identification and robust RL: it reduces uncertainty where possible given a few interactions, but can still act robustly with respect to the remaining uncertainty. On a diverse set of control tasks, our approach demonstrates improved worst-case performance on new environments compared to prior methods based on system identification and on robust RL alone.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑