多智能体强化学习中基于影响范围的信用分配与高效探索
Credit Assignment and Efficient Exploration based on Influence Scope in Multi-agent Reinforcement Learning
AI总结：
针对多智能体强化学习稀疏奖励场景下的信用分配与探索难题，提出ISA算法，通过计算智能体影响范围分配信用并划定探索空间，实验表明其性能优于现有基线。
摘要：
在稀疏奖励场景下训练协作智能体对多智能体强化学习（MARL）构成重大挑战。由于稀疏奖励设置中每一步动作缺乏明确反馈，现有方法难以实现智能体间精确的信用分配与有效探索。本文提出一种解决奖励稀疏域中信用分配和探索问题的新方法，即智能体影响范围（ISA）算法，该算法通过计算单个智能体可影响的状态维度/属性的特定值，确定智能体对状态的影响范围，再利用智能体动作与状态属性间的相互依赖关系，计算信用分配并划定每个智能体的探索空间。我们在多种稀疏奖励多智能体场景中评估ISA，结果显示该方法显著优于现有最先进基线。
英文摘要：
Training cooperative agents in sparse-reward scenarios poses significant challenges for multi-agent reinforcement learning (MARL). Without clear feedback on actions at each step in sparse-reward setting, previous methods struggle with precise credit assignment among agents and effective exploration. In this paper, we introduce a novel method to deal with both credit assignment and exploration problems in reward-sparse domains. Accordingly, we propose an algorithm that calculates the Influence Scope of Agents (ISA) on states by taking specific value of the dimensions/attributes of states that can be influenced by individual agents. The mutual dependence between agents' actions and state attributes are then used to calculate the credit assignment and to delimit the exploration space for each individual agent. We then evaluate ISA in a variety of sparse-reward multi-agent scenarios. The results show that our method significantly outperforms the state-of-art baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
