arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2105.02099cs.AIcs.SYeess.SY
收藏

资源约束下MDP的有效策略综合

Efficient Strategy Synthesis for MDPs with Resource Constraints

Oden Institute, The University of Texas at Austin(德克萨斯大学奥斯汀分校奥登研究所);Faculty of Information Technology, Brno University of Technology(布尔诺理工大学信息技术学院);Faculty of Informatics, Masaryk University(马萨里克大学信息学院);Department of Aerospace Engineering, University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校航空航天工程系);Department of Aerospace Engineering and Engineering Mechanics, The University of Texas at Austin(德克萨斯大学奥斯汀分校航空航天工程与工程力学系)

František Blahoudek, Petr Novotný, Melkior Ornik, Pranay Thangeda, Ufuk Topcu

摘要:

我们认为定性策略合成的形式主义称为消费马尔可夫决策过程。这种形式主义可以在随机环境中的资源约束下运行的代理的动态模型。所提出的算法在时间多项式中工作,相对于模型的表示,它们合成策略,确保以概率1达到给定的一组目标状态(一次或无限次),而不耗尽资源。特别是,当资源量变得太低,无法安全地继续执行任务时,该策略会将代理的过程改变为指定的一组重新加载状态之一,在该状态下,代理将资源分配到满负荷;有足够的资源量,代理会尝试再次完成任务。我们还提出了两种算法,试图减少预期的时间,代理需要完成给定的任务,在实际规划中的一个重要参数。所提出的算法实现和数值例子证明(i)的有效性(在计算时间方面)的规划方法的基础上的消费马尔可夫决策过程和(ii)的积极影响的规划在一个现实的例子。

英文摘要:

We consider qualitative strategy synthesis for the formalism called consumption Markov decision processes. This formalism can model dynamics of an agents that operates under resource constraints in a stochastic environment. The presented algorithms work in time polynomial with respect to the representation of the model and they synthesize strategies ensuring that a given set of goal states will be reached (once or infinitely many times) with probability 1 without resource exhaustion. In particular, when the amount of resource becomes too low to safely continue in the mission, the strategy changes course of the agent towards one of a designated set of reload states where the agent replenishes the resource to full capacity; with sufficient amount of resource, the agent attempts to fulfill the mission again. We also present two heuristics that attempt to reduce expected time that the agent needs to fulfill the given mission, a parameter important in practical planning. The presented algorithms were implemented and numerical examples demonstrate (i) the effectiveness (in terms of computation time) of the planning approach based on consumption Markov decision processes and (ii) the positive impact of the two heuristics on planning in a realistic example.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑