arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2111.09191cs.GTcs.LGcs.MA
收藏

多目标范式博弈中的偏好沟通

Preference Communication in Multi-Objective Normal-Form Games

Vrije Universiteit Brussel(布鲁塞尔自由大学);HU University of Applied Sciences Utrecht(乌得勒支应用科学大学)

Willem Röpke, Diederik M. Roijers, Ann Nowé, Roxana Rădulescu

摘要:

我们考虑两人多目标范式博弈中的偏好交流。在这种博弈中,联合行动的收益是向量值的。以效用为基础的方法,我们假设存在一个效用函数为每个球员向量映射到标量效用,并考虑代理,旨在最大限度地提高预期收益向量的效用。由于智能体通常不知道对手的效用函数或策略,它们必须学习策略才能相互交互。受Stackelberg游戏的启发,我们引入了四种新的偏好通信协议,以帮助代理在到达足够的解决方案。每个协议都描述了一种特定的方法,供一个代理传递对其操作的偏好以及另一个代理如何响应。此外,为了研究何时出现通信,我们引入了一个通信协议,代理必须学习何时进行通信。这些协议随后在一组五个基准游戏上进行评估,以对抗不进行通信的基线代理。我们发现,偏好的沟通可以改变学习过程,并导致出现的循环政策,以前没有观察到在这种情况下。我们进一步观察到,由此产生的政策在很大程度上取决于所玩游戏的特征。最后,我们发现,沟通自然出现在合作和自我利益的设置。

英文摘要:

We consider preference communication in two-player multi-objective normal-form games. In such games, the payoffs resulting from joint actions are vector-valued. Taking a utility-based approach, we assume there exists a utility function for each player which maps vectors to scalar utilities and consider agents that aim to maximise the utility of expected payoff vectors. As agents typically do not know their opponent's utility function or strategy, they must learn policies to interact with each other. Inspired by Stackelberg games, we introduce four novel preference communication protocols to aid agents in arriving at adequate solutions. Each protocol describes a specific approach for one agent to communicate preferences over their actions and how another agent responds. Additionally, to study when communication emerges, we introduce a communication protocol where agents must learn when to communicate. These protocols are subsequently evaluated on a set of five benchmark games against baseline agents that do not communicate. We find that preference communication can alter the learning process and lead to the emergence of cyclic policies which had not been previously observed in this setting. We further observe that the resulting policies can heavily depend on the characteristics of the game that is played. Lastly, we find that communication naturally emerges in both cooperative and self-interested settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑