arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.15780cs.GTcs.AI
收藏

一般博弈中具有人工障碍的纳什均衡自适应学习

Adaptive Learning with Artificial Barriers Yielding Nash Equilibria in General Games

Oslo Metropolitan University(奥斯陆城市大学);Carleton University(卡尔顿大学);University of Agder(阿格德大学)

Ismail Hassan, B. John Oommen, Anis Yazidi

摘要:

学习自动机(LA)中的人工障碍是一个强大但尚未开发的概念,尽管它在20世纪80年代首次提出。引入人工非吸收障碍,使LA计划弹性被困在吸收障碍,这是一种现象,通常被称为锁定概率,导致一个动作的唯一选择收敛后。在LA和一般的强化学习领域中,存在大量的理论工作和具有人工障碍的方案的应用。在本文中,我们设计了一个LA与人工障碍解决一般形式的随机双矩阵对策。经典的LA系统具有吸收屏障的性质,是博弈论中的一个有力工具,在有限信息下收敛到博弈的纳什均衡。然而,LA解决博弈理论问题的工作流只能解决纯策略存在鞍点的情况,而不能达到混合纳什均衡时,纯策略不存在鞍点。在本文中,借助于强大的人工障碍的概念,我们提出了一个LA,收敛到一个最佳的混合纳什均衡,即使可能没有鞍点时,一个纯策略调用。我们部署的计划是线性奖励不作为($L_{R-I}$)的味道,这本来是一个吸收LA计划,但是,我们使其非吸收通过引入人工障碍,在这个意义上,在一个优雅和自然的方式,众所周知的遗产$L_{R-I}$计划可以被看作是我们提出的算法的一个实例,为特定的选择的障碍。此外,我们提出了一个$S$学习版本的LA吸收障碍,能够处理$S$ -学习环境中的反馈是连续的,而不是二进制的情况下的$L_{R-I}$。

英文摘要:

Artificial barriers in Learning Automata (LA) is a powerful and yet under-explored concept although it was first proposed in the 1980s. Introducing artificial non-absorbing barriers makes the LA schemes resilient to being trapped in absorbing barriers, a phenomenon which is often referred to as lock in probability leading to an exclusive choice of one action after convergence. Within the field of LA and reinforcement learning in general, there is a sacristy of theoretical works and applications of schemes with artificial barriers. In this paper, we devise a LA with artificial barriers for solving a general form of stochastic bimatrix game. Classical LA systems possess properties of absorbing barriers and they are a powerful tool in game theory and were shown to converge to game's of Nash equilibrium under limited information. However, the stream of works in LA for solving game theoretical problems can merely solve the case where the Saddle Point of the game exists in a pure strategy and fail to reach mixed Nash equilibrium when no Saddle Point exists for a pure strategy. In this paper, by resorting to the powerful concept of artificial barriers, we suggest a LA that converges to an optimal mixed Nash equilibrium even though there may be no Saddle Point when a pure strategy is invoked. Our deployed scheme is of Linear Reward-Inaction ($L_{R-I}$) flavor which is originally an absorbing LA scheme, however, we render it non-absorbing by introducing artificial barriers in an elegant and natural manner, in the sense that that the well-known legacy $L_{R-I}$ scheme can be seen as an instance of our proposed algorithm for a particular choice of the barrier. Furthermore, we present an $S$ Learning version of our LA with absorbing barriers that is able to handle $S$-Learning environment in which the feedback is continuous and not binary as in the case of the $L_{R-I}$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑