arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.06807cs.GTcs.AIcs.LGstat.ML
收藏

多智能体强化学习中的信息设计

Information Design in Multi-Agent Reinforcement Learning

The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))

Yue Lin, Wenhao Li, Hongyuan Zha, Baoxiang Wang

AI总结:

本文研究多智能体强化学习中的信息设计问题,提出马尔可夫信号博弈框架及信号梯度与扩展服从约束方法,以应对非平稳性和信息被忽略的挑战,并在混合动机任务上验证了算法的高效性。

摘要:

强化学习(RL)的灵感来源于人类婴儿和动物从环境中学习的方式。该设定在某种程度上是理想化的,因为在实际任务中,环境中的其他智能体有各自的目标,并会对自我智能体做出适应性行为。为了在这些环境中良好发展,智能体需要影响其他智能体,使其行为变得更有帮助、更少有害。计算经济学的研究提炼出两种直接影响他人的方式:通过提供有形商品(机制设计)和通过提供信息(信息设计)。本文研究了一组RL智能体的信息设计问题。主要挑战有两个方面。一是所提供的信息会立即影响智能体轨迹的转移,从而引入额外的非平稳性。二是信息可能被忽略,因此发送方必须提供接收方愿意遵从的信息。我们构建了马尔可夫信号博弈,并提出了信号梯度和扩展服从约束的概念来应对这些挑战。我们的算法在各种混合动机任务上表现出高效性,并为计算经济学提供了进一步的见解。我们的代码已在 https://github.com/YueLin301/InformationDesignMARL 公开。

英文摘要:

Reinforcement learning (RL) is inspired by the way human infants and animals learn from the environment. The setting is somewhat idealized because, in actual tasks, other agents in the environment have their own goals and behave adaptively to the ego agent. To thrive in those environments, the agent needs to influence other agents so their actions become more helpful and less harmful. Research in computational economics distills two ways to influence others directly: by providing tangible goods (mechanism design) and by providing information (information design). This work investigates information design problems for a group of RL agents. The main challenges are two-fold. One is the information provided will immediately affect the transition of the agent trajectories, which introduces additional non-stationarity. The other is the information can be ignored, so the sender must provide information that the receiver is willing to respect. We formulate the Markov signaling game, and develop the notions of signaling gradient and the extended obedience constraints that address these challenges. Our algorithm is efficient on various mixed-motive tasks and provides further insights into computational economics. Our code is publicly available at https://github.com/YueLin301/InformationDesignMARL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑