arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.07310eess.SYcs.CCcs.SYmath.OC
收藏

因子化马尔可夫决策过程的最优传感器与执行器选择:复杂性、可近似性及算法

Optimal Sensor and Actuator Selection for Factored Markov Decision Processes: Complexity, Approximability and Algorithms

Elmore Family School of Electrical and Computer Engineering, Purdue University(普渡大学 Elmore 电气与计算机工程学院)

Jayanth Bhargav, Mahsa Ghasemi, Shreyas Sundaram

AI总结:

本文针对因子化马尔可夫决策过程的最优传感器与执行器选择问题,证明其不可近似性,提出贪心算法并验证其在实践中平均可达最优解值70%以上,为相关问题提供了理论与算法支撑。

摘要:

因子化马尔可夫决策过程(fMDPs)是一类马尔可夫决策过程(MDPs),其状态(及动作)可分解为一组状态(及动作)变量,且能通过因子化表示进行紧凑编码。本文考虑fMDP的状态无法直接观测、智能体依赖一组潜在传感器收集信息的场景,在预算有限的情况下,研究选择传感器集合以最大化最优策略提供的无限期折扣回报的问题。我们证明了基础结论:对于任意c>1,该问题无法在n^(1−c)因子内近似,其中n为状态变量数量;传感器选择的不可近似性结果还可推广至一类通用的部分可观测马尔可夫决策过程(POMDPs)。此外,我们研究了预算约束下执行器选择的对偶问题,即设计时选择执行器集合以最大化最优策略下的期望回报,并为此建立了类似的不可近似性结果。最后,我们考虑一种简单的贪心算法,经实验表明,尽管缺乏形式化理论保证,该算法在实际中表现有效,在各类真实世界及随机生成的问题实例中,平均能达到最优解值的70%以上。

英文摘要:

Factored Markov Decision Processes (fMDPs) are a class of Markov Decision Processes (MDPs) in which the states (and actions) can be factored into a set of state (and action) variables and can be encoded compactly using a factored representation. In this paper, we consider a setting where the state of the fMDP is not directly observable, and the agent relies on a set of potential sensors to gather information. We formulate the problem of selecting a set of sensors for fMDPs (under a limited budget) to maximize the infinite-horizon discounted return provided by the optimal policy. We show the fundamental result that it is NP-hard to approximate this problem to within a factor of $n^{1-c}$ for any $c > 1$, where $n$ is the number of state variables. Our inapproximability results for sensor selection also extend to a general class of Partially Observable MDPs (POMDPs). We also consider the dual problem of budgeted actuator selection (at design-time) to maximize the expected return under the optimal policy, for which we establish similar inapproximability results. Finally, we consider a simple greedy algorithm and empirically show that, despite the lack of formal theoretical guarantees, it performs effectively in practice, achieving on average over $70\%$ of the optimal solution value across a variety of real-world and randomly generated problem instances.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑