arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1810.02199math.OC
收藏

概率时态逻辑约束下的近似动态规划

Approximate Dynamic Programming with Probabilistic Temporal Logic Constraints

Lening Li, Jie Fu

摘要:

在本文中,我们发展近似动态规划方法的随机系统建模为马尔可夫决策过程,给定软性能标准和硬约束的一类概率时态逻辑称为概率计算树逻辑(PCTL)。我们的方法包括两个步骤:首先,我们展示了如何将一类PCTL公式转换为机会约束,可以在随机系统规划过程中执行。其次,通过整合随机优化和熵调节动态规划,我们设计了一种新的轨迹采样的近似值迭代法迭代求解的上界的价值函数,同时确保PCTL规格满足的约束。特别地,我们证明了通过对轨迹的策略采样,可以实现近似和真值函数给出的上界之间的紧界。机器人运动规划实例验证了该方法的正确性和有效性。

英文摘要:

In this paper, we develop approximate dynamic programming methods for stochastic systems modeled as Markov Decision Processes, given both soft performance criteria and hard constraints in a class of probabilistic temporal logic called Probabilistic Computation Tree Logic (PCTL). Our approach consists of two steps: First, we show how to transform a class of PCTL formulas into chance constraints that can be enforced during planning in stochastic systems. Second, by integrating randomized optimization and entropy-regulated dynamic programming, we devise a novel trajectory sampling-based approximate value iteration method to iteratively solve for an upper bound on the value function while ensuring the constraints that PCTL specifications are satisfied. Particularly, we show that by the on-policy sampling of the trajectories, a tight bound can be achieved between the upper bound given by the approximation and the true value function. The correctness and efficiency of the method are demonstrated using robotic motion planning examples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑