arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1903.07665math.OC
收藏

部分可观测下的不可预测规划

Unpredictable Planning Under Partial Observability

Michael Hibbard, Yagiz Savas, Bo Wu, Takashi Tanaka, Ufuk Topcu

摘要:

我们研究的问题,合成的控制器,最大化的熵的部分可观察的马尔可夫决策过程(POMDP)的约束下的预期总回报。这样的控制器最大限度地减少了决策者的轨迹的可预测性,同时保证完成由报酬函数表示的任务。首先,我们证明了一个决策者与完美的观察可以随机化的路径至少以及一个决策者与部分观察。然后,专注于有限状态控制器,我们重铸的熵最大化问题作为一个所谓的参数合成问题的参数马尔可夫链(pMC)。我们发现,最大熵的POMDP是下界的最大熵的pMC。最后,我们提出了一个算法,基于一个非线性最优化问题,合成一个FSC,局部最大化的POMDP的FSC的熵与相同数量的内存状态。在数值例子中,我们证明了所提出的算法的运动规划方案。

英文摘要:

We study the problem of synthesizing a controller that maximizes the entropy of a partially observable Markov decision process (POMDP) subject to a constraint on the expected total reward. Such a controller minimizes the predictability of a decision-maker's trajectories while guaranteeing the completion of a task expressed by a reward function. First, we prove that a decision-maker with perfect observations can randomize its paths at least as well as a decision-maker with partial observations. Then, focusing on finite-state controllers, we recast the entropy maximization problem as a so-called parameter synthesis problem for a parametric Markov chain (pMC). We show that the maximum entropy of a POMDP is lower bounded by the maximum entropy of this pMC. Finally, we present an algorithm, based on a nonlinear optimization problem, to synthesize an FSC that locally maximizes the entropy of a POMDP over FSCs with the same number of memory states. In numerical examples, we demonstrate the proposed algorithm on motion planning scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑