一个出奇简单的连续动作 POMDP 求解器：策略树上的惰性交叉熵搜索
A Surprisingly Simple Continuous-Action POMDP Solver: Lazy Cross-Entropy Search Over Policy Trees
AI总结：
针对连续动作 POMDP 在线规划难题，提出 LCEOPT，用惰性交叉熵搜索迭代优化有限时域策略树分布，最高节省两个数量级计算并在高维动作问题上优于现有方法。
摘要：
部分可观测马尔可夫决策过程（POMDP）为随机、部分可观测环境中的决策提供了一个具有原则性的框架。然而，为具有连续动作空间的问题计算良好解仍然具有挑战性。为缓解这一挑战，我们提出了一种简单的在线 POMDP 求解器，称为策略树上的惰性交叉熵搜索（LCEOPT）。在每个规划步骤，我们的方法使用一种新颖的惰性交叉熵方法搜索策略树空间，策略树提供了一种简单的策略表示。具体而言，我们在有前景的有限时域策略树上维护一个分布。该分布通过以下方式迭代更新：采样策略、利用 Monte Carlo 模拟评估策略，并将分布重新拟合到表现最好的策略。我们的方法之所以是惰性的，是因为它利用策略树表示，避免策略采样、评估和分布更新中的冗余计算。这带来了最高达两个数量级的计算节省。与现有最先进方法相比，我们的 LCEOPT 出奇地简单，但在若干连续动作 POMDP 问题上的实验表现优于它们，尤其是在具有更高维动作空间的问题上。
英文摘要：
The Partially Observable Markov Decision Process (POMDP) provides a principled framework for decision making in stochastic partially observable environments. However, computing good solutions for problems with continuous action spaces remains challenging. To ease this challenge, we propose a simple online POMDP solver, called Lazy Cross-Entropy Search Over Policy Trees (LCEOPT). At each planning step, our method uses a novel lazy Cross-Entropy method to search the space of policy trees, which provide a simple policy representation. Specifically, we maintain a distribution on promising finite-horizon policy trees. The distribution is iteratively updated by sampling policies, evaluating them via Monte Carlo simulation, and refitting them to the top-performing ones. Our method is lazy in the sense that it exploits the policy tree representation to avoid redundant computations in policy sampling, evaluation, and distribution update. This leads to computational savings of up to two orders of magnitude. Our LCEOPT is surprisingly simple as compared to existing state-of-the-art methods, yet empirically outperforms them on several continuous-action POMDP problems, particularly for problems with higher-dimensional action spaces.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
