arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.04169cs.AI
收藏

提升动作模型的安全学习

Safe Learning of Lifted Action Models

Washington University in St. Louis(华盛顿大学圣路易斯分校);Palo Alto Research Center(帕洛阿尔托研究中心);Ben Gurion University of the Negev(内盖夫本古里安大学)

Brendan Juba, Hai S. Le, Roni Stern

摘要:

创建一个领域模型,即使是对于经典的、独立于领域的规划,也是一个非常困难的知识工程任务。解决这个问题的自然方法是从观察中学习领域模型。然而,模型学习方法通常不提供安全保证:学习的模型可能会假设动作是适用的,而实际上它们并不适用,并且可能会错误地捕捉动作的效果。这可能导致生成在执行时失败的计划。在某些领域,由于失败的成本或失败后无法在线重新计划,这种失败是不可接受的。在这样的设置中,所有的学习都必须基于收集的一些观察来离线完成,例如,被其他特工或人类杀死通过这种学习,任务是生成一个保证成功的计划。这就是所谓的无模型规划问题。先前的工作提出了一种算法来解决经典规划中的无模型规划问题。然而,他们仅限于学习基础领域,因此无法扩展。我们推广了这项工作,并提出了第一个安全的无模型规划算法提升域。我们证明了我们的方法的正确性,并提供了一个统计分析表明,需要解决未来的问题,高概率的轨迹的数量是线性的潜在大小的领域模型。我们还提出了12 IPC域上的实验表明,我们的方法是能够学习真正的动作模型在所有情况下,最多两个轨迹。

英文摘要:

Creating a domain model, even for classical, domain-independent planning, is a notoriously hard knowledge-engineering task. A natural approach to solve this problem is to learn a domain model from observations. However, model learning approaches frequently do not provide safety guarantees: the learned model may assume actions are applicable when they are not, and may incorrectly capture actions' effects. This may result in generating plans that will fail when executed. In some domains such failures are not acceptable, due to the cost of failure or inability to replan online after failure. In such settings, all learning must be done offline, based on some observations collected, e.g., by some other agents or a human. Through this learning, the task is to generate a plan that is guaranteed to be successful. This is called the model-free planning problem. Prior work proposed an algorithm for solving the model-free planning problem in classical planning. However, they were limited to learning grounded domains, and thus they could not scale. We generalize this prior work and propose the first safe model-free planning algorithm for lifted domains. We prove the correctness of our approach, and provide a statistical analysis showing that the number of trajectories needed to solve future problems with high probability is linear in the potential size of the domain model. We also present experiments on twelve IPC domains showing that our approach is able to learn the real action model in all cases with at most two trajectories.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑