马尔可夫决策模型最优值对转移概率函数偏差的一阶灵敏度
First-order sensitivity of the optimal value in a Markov decision model with respect to deviations in the transition probability function
摘要:
实际应用中使用的马尔可夫决策模型(MDM)通常没有底层的“真正”MDM复杂。模型复杂性的降低是出于几个原因。然而,很明显,了解哪种模型简化是合理的(关于最优值),哪种是不合理的是有意义的。在这篇文章中,我们提出了一种方法来解决这个问题。我们引入了一种最优值的导数作为转移概率的函数,它可以用来度量最优值对转移概率的(一阶)灵敏度。转移概率的变化。“可微性”是为一个相当广泛的MDM类获得的,“导数”是明确规定的。我们的理论研究结果说明了库存控制和数学金融中的最优化问题。
英文摘要:
Markov decision models (MDM) used in practical applications are most often less complex than the underlying `true' MDM. The reduction of model complexity is performed for several reasons. However, it is obviously of interest to know what kind of model reduction is reasonable (in regard to the optimal value) and what kind is not. In this article we propose a way how to address this question. We introduce a sort of derivative of the optimal value as a function of the transition probabilities, which can be used to measure the (first-order) sensitivity of the optimal value w.r.t.\ changes in the transition probabilities. `Differentiability' is obtained for a fairly broad class of MDMs, and the `derivative' is specified explicitly. Our theoretical findings are illustrated by means of optimization problems in inventory control and mathematical finance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。