马尔可夫决策过程上的正性困难性结果
Positivity-hardness results on Markov decision processes
AI总结：
本文通过从线性递推序列 Positivity 问题的多项式时间归约，证明单计数器 MDPs 和整数权重 MDPs 上的一系列优化问题在数学上本质困难，其算法求解需解析数论的重大突破。
摘要：
本文研究了一类具有有限状态空间的单计数器马尔可夫决策过程（MDPs）和整数权重 MDPs 的一系列优化问题。具体而言，本文考虑了以下问题：单计数器 MDPs 的终止概率和期望终止时间，以及整数权重 MDPs 的能量目标满足概率、条件期望和部分期望、总累积权重约束的满足概率、累积权重的分位数计算，以及累积权重的条件风险价值（conditional value-at-risk）。尽管对于某些特殊情形已有算法结果，但这些问题的判定版本在一般情况下是否可判定仍是未知的。本文通过提供从线性递推序列的 Positivity 问题的多项式时间归约，证明了这些优化问题在数学上本质上是困难的。Positivity 问题是一个著名的数论问题，其可判定性状态数十年来一直悬而未决，并且已知 Positivity 问题的可判定性将对解析数论产生深远影响。因此，本文给出的归约表明，若没有解析数论上的重大突破，就不可能为所研究的任何一个问题找到算法解。这些归约依赖于构造能够编码线性递推序列初始值和线性递推关系的 MDP 构件（MDP-gadgets）。这些构件可以灵活调整，以证明各种 Positivity 困难性结果。
英文摘要：
This paper investigates a series of optimization problems for one-counter Markov decision processes (MDPs) and integer-weighted MDPs with finite state space. Specifically, it considers problems addressing termination probabilities and expected termination times for one-counter MDPs, as well as satisfaction probabilities of energy objectives, conditional and partial expectations, satisfaction probabilities of constraints on the total accumulated weight, the computation of quantiles for the accumulated weight, and the conditional value-at-risk for accumulated weights for integer-weighted MDPs. Although algorithmic results are available for some special instances, the decidability status of the decision versions of these problems is unknown in general. The paper demonstrates that these optimization problems are inherently mathematically difficult by providing polynomial-time reductions from the Positivity problem for linear recurrence sequences. This problem is a well-known number-theoretic problem whose decidability status has been open for decades and it is known that decidability of the Positivity problem would have far-reaching consequences in analytic number theory. So, the reductions presented in the paper show that an algorithmic solution to any of the investigated problems is not possible without a major breakthrough in analytic number theory. The reductions rely on the construction of MDP-gadgets that encode the initial values and linear recurrence relations of linear recurrence sequences. These gadgets can flexibly be adjusted to prove the various Positivity-hardness results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
