可扩展的多智能体强化学习与集体影响估计
Scalable Multiagent Reinforcement Learning with Collective Influence Estimation
AI总结：
本文提出了一种可扩展的多智能体强化学习框架，通过集体影响估计网络实现高效协作，避免网络扩展并提升鲁棒性和部署可行性。
摘要：
多智能体强化学习（MARL）因其在解决复杂协作任务中的潜力而受到广泛关注。然而，现有的MARL方法往往依赖于智能体之间频繁交换动作或状态信息以实现有效的协调，这在实际的机器人系统中难以满足。常见的解决方案是引入估计网络来建模其他智能体的行为并预测其动作；然而，此类设计导致估计网络的大小和计算成本随着智能体数量的增加而迅速增长，从而限制了大规模系统中的可扩展性。为了解决这些挑战，本文提出了一种多智能体学习框架，其中加入了集体影响估计网络（CIEN）。通过显式建模其他智能体对任务对象的集体影响，每个智能体可以仅从其本地观察和任务对象的状态推断出关键的交互信息，从而在不进行显式动作信息交换的情况下实现高效的协作。所提出的框架在团队规模增加时有效避免了网络扩展；此外，新智能体可以被纳入而无需修改现有智能体的网络结构，展示了强大的可扩展性。基于Soft Actor-Critic（SAC）算法的多智能体协作任务的实验结果表明，所提出的方法在通信受限的环境中实现了稳定且高效的协调。此外，使用集体影响建模训练的策略在真实机器人平台上部署，实验结果表明显著提高了鲁棒性和部署可行性，并减少了对通信基础设施的依赖。
英文摘要：
Multiagent reinforcement learning (MARL) has attracted considerable attention due to its potential in addressing complex cooperative tasks. However, existing MARL approaches often rely on frequent exchanges of action or state information among agents to achieve effective coordination, which is difficult to satisfy in practical robotic systems. A common solution is to introduce estimator networks to model the behaviors of other agents and predict their actions; nevertheless, such designs cause the size and computational cost of the estimator networks to grow rapidly with the number of agents, thereby limiting scalability in large-scale systems. To address these challenges, this paper proposes a multiagent learning framework augmented with a Collective Influence Estimation Network (CIEN). By explicitly modeling the collective influence of other agents on the task object, each agent can infer critical interaction information solely from its local observations and the task object's states, enabling efficient collaboration without explicit action information exchange. The proposed framework effectively avoids network expansion as the team size increases; moreover, new agents can be incorporated without modifying the network structures of existing agents, demonstrating strong scalability. Experimental results on multiagent cooperative tasks based on the Soft Actor-Critic (SAC) algorithm show that the proposed method achieves stable and efficient coordination under communication-limited environments. Furthermore, policies trained with collective influence modeling are deployed on a real robotic platform, where experimental results indicate significantly improved robustness and deployment feasibility, along with reduced dependence on communication infrastructure.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
