arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.09038eess.SYcs.SY
收藏

具有有限线性时序逻辑约束的部分可观测Markov决策过程的最优控制

Optimal Control of Partially Observable Markov Decision Processes with Finite Linear Temporal Logic Constraints

Krishna C. Kalagarla, Dhruva Kartik, Dongming Shen, Rahul Jain, Ashutosh Nayyar, Pierluigi Nuzzo

摘要:

自治代理通常在状态被部分观察的情况下运行。除了最大化他们的累积回报,代理必须执行复杂的任务,具有丰富的时间和逻辑结构。这些任务可以使用时态逻辑语言来表达,如有限线性时态逻辑(LTL_f)。本文,第一次,提供了一个结构化的框架设计代理政策,最大限度地提高回报,同时确保满足时序逻辑规范的概率是足够高的。我们将该问题转化为一个受约束的部分可观测马尔可夫决策过程(POMDP),并提出了一种新的方法,该方法可以利用现成的无约束POMDP求解器来求解该问题,保证了近似最优性和高概率的约束满足。我们通过在几个感兴趣的模型上实现它来证明它的有效性。

英文摘要:

Autonomous agents often operate in scenarios where the state is partially observed. In addition to maximizing their cumulative reward, agents must execute complex tasks with rich temporal and logical structures. These tasks can be expressed using temporal logic languages like finite linear temporal logic (LTL_f). This paper, for the first time, provides a structured framework for designing agent policies that maximize the reward while ensuring that the probability of satisfying the temporal logic specification is sufficiently high. We reformulate the problem as a constrained partially observable Markov decision process (POMDP) and provide a novel approach that can leverage off-the-shelf unconstrained POMDP solvers for solving it. Our approach guarantees approximate optimality and constraint satisfaction with high probability. We demonstrate its effectiveness by implementing it on several models of interest.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑