arXiv 1301.04512013-01-03math.OC

动态规划中的一般极限值

General limit value in Dynamic Programming

Jérôme Renault

摘要：

考虑了一个具有任意状态空间和有界报酬的动态规划问题。是否有可能以一种独特的方式定义一个极限值的问题，其中的“耐心”的决策者往往是无限的？我们认为，对于每个评估$θ$（概率分布在正整数）的值函数$v_θ$的问题，其中的任何阶段$t$的权重是由$θ_t$，我们调查的一致收敛序列$(v_{θ^k})_k$时的“不耐烦”的评价消失，在这个意义上，$\sum_{t} |θ^k_{t}-θ^k_{t+1}| \rightarrow_{k \to \infty} 0$。我们证明了这种一致收敛的发生当且仅当度量空间${v_{θ^k}, k\geq 1}$是完全有界的。此外，存在一个特定的函数$v^*$，它与特定的选择序列$({θ^k})_k$无关，使得这种值函数序列的任何极限点都精确地是$v^*$。因此，当谈到一致收敛的价值函数，$v^*$可以被认为是唯一的可能的限制时，耐心的决策者趋于无穷大。结果特别适用于贴现因子为零时的贴现收益，以及阶段数趋于无穷大时的平均收益，也适用于具有随机转移的模型。我们提出易于处理的推论，我们讨论反例和猜想。

英文摘要：

We consider a dynamic programming problem with arbitrary state space and bounded rewards. Is it possible to define in an unique way a limit value for the problem, where the "patience" of the decision-maker tends to infinity ? We consider, for each evaluation $θ$ (a probability distribution over positive integers) the value function $v_θ$ of the problem where the weight of any stage $t$ is given by $θ_t$, and we investigate the uniform convergence of a sequence $(v_{θ^k})_k$ when the "impatience" of the evaluations vanishes, in the sense that $\sum_{t} |θ^k_{t}-θ^k_{t+1}| \rightarrow_{k \to \infty} 0$. We prove that this uniform convergence happens if and only if the metric space ${v_{θ^k}, k\geq 1}$ is totally bounded. Moreover there exists a particular function $v^*$, independent of the particular chosen sequence $({θ^k})_k$, such that any limit point of such sequence of value functions is precisely $v^*$. Consequently, while speaking of uniform convergence of the value functions, $v^*$ may be considered as the unique possible limit when the patience of the decision-maker tends to infinity. The result applies in particular to discounted payoffs when the discount factor vanishes, as well as to average payoffs where the number of stages goes to infinity, and also to models with stochastic transitions. We present tractable corollaries, and we discuss counterexamples and a conjecture.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文