对于大类的POMDP，不需要未来的存储器
Future memories are not needed for large classes of POMDPs
摘要：
部分观测马尔可夫决策过程（POMDP）的最优策略是历史依赖的：决策是基于整个观测历史做出的。无记忆策略，仅根据最后一次观察做出决策，在文献中通常被认为是无用的，因为我们可以构建POMDP实例，其中最佳无记忆策略任意地比历史依赖策略更差。我们的目标是挑战这种信念。我们表明，最佳的无记忆策略可以有效地使用混合整数线性规划（MILP）计算，并执行合理的范围广泛的情况下，从文献。当用有效的不等式加强时，该MILP的线性松弛提供了最优历史相关策略的值的高质量上界。此外，当使用有限时域POMDP问题与无记忆策略作为滚动最优化问题时，模型预测控制方法导致有效的历史相关策略，我们称之为未来短记忆（SMF）策略。基本上，SMF策略利用这些无记忆策略来构建Bellman值函数的近似值。数值实验表明，我们的方法的效率从文献中的基准实例。
英文摘要：
Optimal policies for partially observed Markov decision processes (POMDPs) are history-dependent: Decisions are made based on the entire history of observation. Memoryless policies, which take decisions based on the last observation only, are generally considered useless in the literature because we can construct POMDP instances for which optimal memoryless policies are arbitrarily worse than history-dependent ones. Our purpose is to challenge this belief. We show that optimal memoryless policies can be computed efficiently using mixed integer linear programming (MILP), and perform reasonably well on a wide range of instances from the literature. When strengthened with valid inequalities, the linear relaxation of this MILP provides high quality upper-bounds on the value of an optimal history dependent policy. Furthermore, when used with a finite horizon POMDP problem with memoryless policies as rolling optimization problem, a model predictive control approach leads to an efficient history-dependent policy, which we call the short memory in the future (SMF) policy. Basically, the SMF policy leverages these memoryless policies to build an approximation of the Bellman value function. Numerical experiments show the efficiency of our approach on benchmark instances from the literature.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
