ColorGrid:用于目标推断与辅助的多智能体非平稳环境
ColorGrid: A Multi-Agent Non-Stationary Environment for Goal Inference and Assistance
AI总结:
本文提出ColorGrid多智能体非平稳环境,用于测试目标推断与辅助,发现IPPO算法在非平稳和不对称目标下无法解决该问题,并开源代码以支持基准测试。
摘要:
自主智能体与人类的交互日益聚焦于适应人类不断变化的偏好,以提升在现实任务中的辅助能力。有效的智能体必须学会准确推断人类目标(这些目标往往是隐藏的),才能实现良好协作。然而,现有的多智能体强化学习(MARL)环境缺乏严格评估这些智能体学习能力所必需的属性。为此,我们提出了ColorGrid,一种新颖的MARL环境,具有可定制的非平稳性、不对称性和奖励结构。我们研究了独立近端策略优化(IPPO)——一种最先进的(SOTA)MARL算法——在ColorGrid中的性能,并通过大量消融实验发现,特别是在代表人类的“领导者”智能体与“跟随者”辅助智能体之间存在同时非平稳且不对称的目标时,IPPO无法解决ColorGrid问题。为支持未来MARL算法的基准测试,我们在https://github.com/andreyrisukhin/ColorGrid上发布了我们的环境代码、模型检查点和轨迹可视化。
英文摘要:
Autonomous agents' interactions with humans are increasingly focused on adapting to their changing preferences in order to improve assistance in real-world tasks. Effective agents must learn to accurately infer human goals, which are often hidden, to collaborate well. However, existing Multi-Agent Reinforcement Learning (MARL) environments lack the necessary attributes required to rigorously evaluate these agents' learning capabilities. To this end, we introduce ColorGrid, a novel MARL environment with customizable non-stationarity, asymmetry, and reward structure. We investigate the performance of Independent Proximal Policy Optimization (IPPO), a state-of-the-art (SOTA) MARL algorithm, in ColorGrid and find through extensive ablations that, particularly with simultaneous non-stationary and asymmetric goals between a ``leader'' agent representing a human and a ``follower'' assistant agent, ColorGrid is unsolved by IPPO. To support benchmarking future MARL algorithms, we release our environment code, model checkpoints, and trajectory visualizations at https://github.com/andreyrisukhin/ColorGrid.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。