解质量不打折：通过自适应多级简化加速信念依赖的连续POMDP
No Compromise in Solution Quality: Speeding Up Belief-dependent Continuous POMDPs via Adaptive Multilevel Simplification
AI总结：
本文提出自适应多级简化理论，加速信念依赖的连续POMDP在线规划，在不牺牲解质量的前提下，通过新型上下界替代信息论奖励计算，实现显著加速。
摘要：
具有一般信念依赖奖励的连续POMDP在线求解是出了名的困难。在本文中，我们针对给定外部构建的信念树以及使用探索技术在线构建信念树的MCTS设置，提出了自适应多级简化的完整可证明理论。我们的理论允许在不牺牲所得解质量的情况下加速具有信念依赖奖励的POMDP规划。我们严格证明了所提出的统一理论中的每一个理论主张。利用一般理论结果，我们提出了三种算法来加速具有信念依赖奖励的连续POMDP在线规划。我们的两种算法SITH-BSP和LAZY-SITH-BSP可以用于任何外部构建信念树的方法之上。第三种算法SITH-PFT是一种随时MCTS方法，允许插入任何探索技术。所有方法都保证返回与其未简化版本完全相同的最优动作。我们用本文中推导的新的自适应上下界替换了代价高昂的信息论奖励计算，这些界本身也具有独立意义。我们证明它们易于计算，并且可以根据算法的需求进行收紧。我们的方法是通用的；即，任何单调收敛于奖励的界都可以用来实现显著的加速，而不会损失任何性能。我们的理论和算法支持连续状态、动作和观测的具有挑战性的设置。信念可以是参数化的或一般的，并由加权粒子表示。我们在仿真中展示了与基线方法相比，在保证相同性能的情况下，规划速度显著提升。
英文摘要：
Continuous POMDPs with general belief-dependent rewards are notoriously difficult to solve online. In this paper, we present a complete provable theory of adaptive multilevel simplification for the setting of a given externally constructed belief tree and MCTS that constructs the belief tree on the fly using an exploration technique. Our theory allows to accelerate POMDP planning with belief-dependent rewards without any sacrifice in the quality of the obtained solution. We rigorously prove each theoretical claim in the proposed unified theory. Using the general theoretical results, we present three algorithms to accelerate continuous POMDP online planning with belief-dependent rewards. Our two algorithms, SITH-BSP and LAZY-SITH-BSP, can be utilized on top of any method that constructs a belief tree externally. The third algorithm, SITH-PFT, is an anytime MCTS method that permits to plug-in any exploration technique. All our methods are guaranteed to return exactly the same optimal action as their unsimplified equivalents. We replace the costly computation of information-theoretic rewards with novel adaptive upper and lower bounds which we derive in this paper, and are of independent interest. We show that they are easy to calculate and can be tightened by the demand of our algorithms. Our approach is general; namely, any bounds that monotonically converge to the reward can be utilized to achieve significant speedup without any loss in performance. Our theory and algorithms support the challenging setting of continuous states, actions, and observations. The beliefs can be parametric or general and represented by weighted particles. We demonstrate in simulation a significant speedup in planning compared to baseline approaches with guaranteed identical performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
