arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1907.06128math.OCcs.SYeess.SY
收藏

具有受控观测的连续时间Markov决策过程

Continuous-Time Markov Decision Processes with Controlled Observations

Yunhan Huang, Veeraruna Kavitha, Quanyan Zhu

摘要:

本文研究了具有受控行为和观测值的连续时间折扣跳马尔可夫决策过程。该观测仅可用于时间实例的离散集。在每次观测时,必须为下一次观测选择最佳时机,并为两个观测点之间的时间间隔选择控制轨迹。我们提供了一个理论框架,决策者可以利用找到最佳的观察时期和最佳的行动联合。调查了两起案件。一个是门控混沌系统,其中我们明确地描述了最优行动和最优观测,其中最优观测被证明是独立的状态。另一个是具有Poisson到达过程的库存控制问题,我们用数值方法得到了最优行动和最优观测。结果表明,在最佳动作不断适应的状态区域,更频繁地观察是最佳的。

英文摘要:

In this paper, we study a continuous-time discounted jump Markov decision process with both controlled actions and observations. The observation is only available for a discrete set of time instances. At each time of observation, one has to select an optimal timing for the next observation and a control trajectory for the time interval between two observation points. We provide a theoretical framework that the decision maker can utilize to find the optimal observation epochs and the optimal actions jointly. Two cases are investigated. One is gated queueing systems in which we explicitly characterize the optimal action and the optimal observation where the optimal observation is shown to be independent of the state. Another is the inventory control problem with Poisson arrival process in which we obtain numerically the optimal action and observation. The results show that it is optimal to observe more frequently at a region of states where the optimal action adapts constantly.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑