arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2110.14529cs.GTcs.AIcs.LG
收藏

利用凹凸性和Lipschitz性质的zs-POSG的HSVI

HSVI for zs-POSGs using Concavity, Convexity and Lipschitz Properties

Univ. Lyon(里昂大学);INSA Lyon(里昂国立应用科学学院);INRIA(法国国家信息与自动化研究所);Université de Lorraine(洛林大学);CNRS(法国国家科学研究中心)

Aurélien Delage, Olivier Buffet, Jilles Dibangoye

摘要:

动态规划和启发式搜索是最先进的顺序决策问题求解器的核心。在部分可观察或协作设置(\eg,POMDP和Dec-POMDP),这需要引入一个适当的统计,诱导一个完全可观察的问题,以及边界(凸)逼近的最佳价值函数。这种方法在2-player零和部分可观察随机博弈(zs-POSGs)的一些子类中也取得了成功,但在一般情况下失败了,尽管已知的凸性和凸性,这只会导致启发式算法的收敛性差的保证。我们克服了这个问题,利用这些属性来获得边界逼近和有效的更新和选择算子,然后推导出一个原型求解器的启发HSVI,可证明收敛到一个$ε$ -最佳的解决方案,在有限的时间内,我们凭经验评估。这打开了一扇大门,一个新的家庭有前途的方法,补充那些依赖于线性规划或迭代法。

英文摘要:

Dynamic programming and heuristic search are at the core of state-of-the-art solvers for sequential decision-making problems. In partially observable or collaborative settings (\eg, POMDPs and Dec-POMDPs), this requires introducing an appropriate statistic that induces a fully observable problem as well as bounding (convex) approximators of the optimal value function. This approach has succeeded in some subclasses of 2-player zero-sum partially observable stochastic games (zs-POSGs) as well, but failed in the general case despite known concavity and convexity properties, which only led to heuristic algorithms with poor convergence guarantees. We overcome this issue, leveraging on these properties to derive bounding approximators and efficient update and selection operators, before deriving a prototypical solver inspired by HSVI that provably converges to an $ε$-optimal solution in finite time, and which we empirically evaluate. This opens the door to a novel family of promising approaches complementing those relying on linear programming or iterative methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑