arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.08755cs.AI
收藏

在JESP中使用有限状态控制器求解无限时域Dec-POMDP

Solving infinite-horizon Dec-POMDPs using Finite State Controllers within JESP

Université de Lorraine(洛林大学);INRIA(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);LORIA(洛林计算机科学与应用数学实验室)

Yang You, Vincent Thomas, Francis Colas, Olivier Buffet

摘要:

本文着眼于通过搜索纳什均衡来解决形式化为分散式POMDP(Dec-POMDP)的协同规划问题,即,每个代理的策略是对其他代理的(固定)策略的最佳响应的情况。虽然联合基于平衡的搜索策略(JESP)算法在有限时域设置依赖于策略树,我们建议在这里适应它的无限时域Dec-POMDPs通过使用有限状态控制器(FSC)的政策表示。在这篇文章中,我们(1)解释了如何将具有$N-1$固定FSC的Dec-POMDP转化为解是$N^\text{th}$代理最佳对策的无限时域POMDP;(2)提出了一个JESP变体,称为\infJESP,使用它来求解无限时域Dec-POMDP;(3)引入JESP的启发式初始化,旨在得到好的解;以及(4)在最先进的基准问题上进行实验以评估我们的方法。

英文摘要:

This paper looks at solving collaborative planning problems formalized as Decentralized POMDPs (Dec-POMDPs) by searching for Nash equilibria, i.e., situations where each agent's policy is a best response to the other agents' (fixed) policies. While the Joint Equilibrium-based Search for Policies (JESP) algorithm does this in the finite-horizon setting relying on policy trees, we propose here to adapt it to infinite-horizon Dec-POMDPs by using finite state controller (FSC) policy representations. In this article, we (1) explain how to turn a Dec-POMDP with $N-1$ fixed FSCs into an infinite-horizon POMDP whose solution is an $N^\text{th}$ agent best response; (2) propose a JESP variant, called \infJESP, using this to solve infinite-horizon Dec-POMDPs; (3) introduce heuristic initializations for JESP aiming at leading to good solutions; and (4) conduct experiments on state-of-the-art benchmark problems to evaluate our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑