可数马尔可夫决策过程中点收益、平均收益和总收益目标的策略复杂性
Strategy Complexity of Point Payoff, Mean Payoff and Total Payoff Objectives in Countable MDPs
AI总结：
本文研究了可数MDPs中点收益、平均收益和总收益目标的策略复杂性，确定了实现ε-最优或最优策略所需的内存，部分情况仅需无记忆确定性策略，部分需计数器。
摘要：
我们研究具有实值转移奖励的可数无限马尔可夫决策过程（MDPs）。每个无限运行诱导以下收益序列：1. 点收益（直接观察到的转移奖励序列），2. 平均收益（迄今为止所有奖励之和除以步数的序列），以及 3. 总收益（迄今为止所有奖励之和的序列）。对于每种收益类型，目标是最大化 $\liminf$ 非负的概率。我们建立了这些目标的策略复杂性的完整图景，即对于 $\varepsilon$-最优（分别地，最优）策略，多少内存是必要且充分的。某些情况可以通过无记忆确定性策略获胜，而其他情况则需要步数计数器、奖励计数器或两者兼需。
英文摘要：
We study countably infinite Markov decision processes (MDPs) with real-valued transition rewards. Every infinite run induces the following sequences of payoffs: 1. Point payoff (the sequence of directly seen transition rewards), 2. Mean payoff (the sequence of the sums of all rewards so far, divided by the number of steps), and 3. Total payoff (the sequence of the sums of all rewards so far). For each payoff type, the objective is to maximize the probability that the $\liminf$ is non-negative. We establish the complete picture of the strategy complexity of these objectives, i.e., how much memory is necessary and sufficient for $\varepsilon$-optimal (resp. optimal) strategies. Some cases can be won with memoryless deterministic strategies, while others require a step counter, a reward counter, or both.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
