PIMAEX:通过同伴激励实现多智能体探索
PIMAEX: Multi-Agent Exploration through Peer Incentivization
AI总结:
本文提出同伴激励多智能体探索奖励PIMAEX及通信训练算法PIMAEX-Communication,通过鼓励智能体相互影响以提升新颖状态探索,并在具有欺骗性奖励的部分可观测环境中验证其优于基线方法。
摘要:
尽管近年来单智能体强化学习中的探索问题已得到广泛研究,但针对多智能体强化学习中对应问题的研究却少得多。为解决这一问题,本文受先前关于内在好奇心和基于影响力的奖励研究启发,提出了一种同伴激励奖励函数。PIMAEX奖励,全称为同伴激励多智能体探索(Peer-Incentivized Multi-Agent Exploration),旨在通过鼓励智能体相互施加影响力,以提高遇到新颖状态的可能性,从而改善多智能体环境中的探索。我们将PIMAEX奖励与PIMAEX-Communication结合进行评估,后者是一种多智能体训练算法,利用通信通道使智能体能够相互影响。评估在Consume/Explore环境中进行,这是一个具有欺骗性奖励的部分可观测环境,专门设计用于挑战探索与利用困境以及信用分配问题。实验结果实证表明,使用PIMAEX奖励与PIMAEX-Communication的智能体优于未使用该组合的智能体。
英文摘要:
While exploration in single-agent reinforcement learning has been studied extensively in recent years, considerably less work has focused on its counterpart in multi-agent reinforcement learning. To address this issue, this work proposes a peer-incentivized reward function inspired by previous research on intrinsic curiosity and influence-based rewards. The \textit{PIMAEX} reward, short for Peer-Incentivized Multi-Agent Exploration, aims to improve exploration in the multi-agent setting by encouraging agents to exert influence over each other to increase the likelihood of encountering novel states. We evaluate the \textit{PIMAEX} reward in conjunction with \textit{PIMAEX-Communication}, a multi-agent training algorithm that employs a communication channel for agents to influence one another. The evaluation is conducted in the \textit{Consume/Explore} environment, a partially observable environment with deceptive rewards, specifically designed to challenge the exploration vs.\ exploitation dilemma and the credit-assignment problem. The results empirically demonstrate that agents using the \textit{PIMAEX} reward with \textit{PIMAEX-Communication} outperform those that do not.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。