arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.18632eess.SYcs.SY
收藏

带有线性时序逻辑任务的马尔可夫决策过程效率最优控制

Optimal Control of Markov Decision Processes for Efficiency with Linear Temporal Logic Tasks

Yu Chen, Xuanyuan Yin, Shaoyuan Li, Xiang Yin

AI总结:

研究马尔可夫决策过程在满足线性时序逻辑定性任务前提下的效率最优控制问题,通过结合状态分类与扰动分析提出一种实现$ε$-最优性的平稳控制策略综合方法。

摘要:

我们研究马尔可夫决策过程(MDPs)的最优控制综合问题,同时处理定性与定量目标。具体而言,我们要求系统以概率1满足由线性时序逻辑(LTL)公式指定的定性任务。此外,为量化系统性能,我们引入效率概念,定义为奖励与成本的比率。该度量比标准的长期平均奖励指标更通用,因其旨在最大化每单位成本所获奖励。我们的目标是综合一种控制策略,不仅确保满足LTL任务,还能最大化效率。我们提出了一种有效方法,通过以新颖方式将MDPs的状态分类与扰动分析相结合,来综合实现$ε$-最优性的平稳控制策略。我们的结果通过纳入定性LTL任务,扩展了关于MDPs效率最优控制综合的现有工作。提供了机器人任务规划中的案例研究以说明所提出的算法。

英文摘要:

We investigate the problem of optimal control synthesis for Markov Decision Processes (MDPs), addressing both qualitative and quantitative objectives. Specifically, we require the system to satisfy a qualitative task specified by a Linear Temporal Logic (LTL) formula with probability one. Additionally, to quantify the system's performance, we introduce the concept of efficiency, defined as the ratio between rewards and costs. This measure is more general than the standard long-run average reward metric, as it seeks to maximize the reward obtained per unit cost. Our objective is to synthesize a control policy that not only ensures the LTL task is satisfied but also maximizes efficiency. We present an effective approach for synthesizing a stationary control policy that achieves $ε$-optimality by integrating state classifications of MDPs with perturbation analysis in a novel manner. Our results extend existing work on efficiency-optimal control synthesis for MDPs by incorporating qualitative LTL tasks. Case studies in robot task planning are provided to illustrate the proposed algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑