arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.03207cs.LG
收藏

基于概念调制模型的离线强化学习快速泛化

Concept-modulated model-based offline reinforcement learning for rapid generalization

HRL Laboratories(HRL实验室)

Nicholas A. Ketz, Praveen K. Pilly

摘要:

任何机器学习解决方案的鲁棒性从根本上都受到其训练数据的约束。超越原始训练的一种推广方法是通过原始数据集的人工信息增强;然而,不可能指定部署过程中可能发生的所有可能的故障情况。为了解决这一限制,我们结合了基于模型的强化学习和模型可解释性方法,提出了一种解决方案,该解决方案可以自动生成受环境概念和以无监督方式学习的动态约束的模拟场景。特别是,代理的环境的内部模型的条件是对代理的动作敏感的输入空间的低维概念表示。我们证明了这种方法在一个标准的现实驾驶模拟器在一个简单的点到点导航任务,在那里我们表现出显着的改善,在一杆概括到不同的情况下,指定的故障情况下,以及zero-shot概括到类似的变化相比,基于模型和无模型的方法。

英文摘要:

The robustness of any machine learning solution is fundamentally bound by the data it was trained on. One way to generalize beyond the original training is through human-informed augmentation of the original dataset; however, it is impossible to specify all possible failure cases that can occur during deployment. To address this limitation we combine model-based reinforcement learning and model-interpretability methods to propose a solution that self-generates simulated scenarios constrained by environmental concepts and dynamics learned in an unsupervised manner. In particular, an internal model of the agent's environment is conditioned on low-dimensional concept representations of the input space that are sensitive to the agent's actions. We demonstrate this method within a standard realistic driving simulator in a simple point-to-point navigation task, where we show dramatic improvements in one-shot generalization to different instances of specified failure cases as well as zero-shot generalization to similar variations compared to model-based and model-free approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑