arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.02516cs.MAcs.LG
收藏

多智能体环境中跨独立强化学习智能体的交互模式涌现学习

Learning Emergence of Interaction Patterns across Independent RL Agents in Multi-Agent Environments

Virginia Tech(弗吉尼亚理工大学);Siemens Technology(西门子技术公司)

Vasanth Reddy Baddam, Suat Gumussoy, Almuatazbellah Boker, Hoda Eldardiry

AI总结:

本文提出Bottom Up Network(BUN),将多智能体集合视为统一实体并通过专门权重初始化和基于梯度的稀疏动态连接实现独立学习与按需协调,在合作导航和交通控制等任务中以更低计算成本超越基线方法。

摘要:

许多现实世界问题,例如控制无人机群和城市交通,天然适合建模为多智能体强化学习(RL)问题。然而,现有的多智能体RL方法常常面临可扩展性挑战,主要原因是智能体之间引入了通信。因此,一个关键挑战在于将单智能体RL中深度学习的成功经验适配到多智能体环境中。为应对这一挑战,我们提出了一种从根本上重新构想多智能体环境的方法。与传统方法为每个智能体分别建模并使用独立网络不同,我们的方法——Bottom Up Network(BUN)——采用了一种独特的视角。BUN将多智能体集合视为一个统一实体,同时采用专门的权重初始化策略来促进独立学习。此外,我们利用梯度信息动态建立智能体之间的连接,在必要时实现协调,同时保持这些连接有限且稀疏,以有效管理计算预算。我们在多种合作型多智能体场景中进行了广泛的实证评估,包括合作导航和交通控制等任务,结果一致表明BUN在显著降低计算成本的同时优于基线方法。

英文摘要:

Many real-world problems, such as controlling swarms of drones and urban traffic, naturally lend themselves to modeling as multi-agent reinforcement learning (RL) problems. However, existing multi-agent RL methods often suffer from scalability challenges, primarily due to the introduction of communication among agents. Consequently, a key challenge lies in adapting the success of deep learning in single-agent RL to the multi-agent setting. In response to this challenge, we propose an approach that fundamentally reimagines multi-agent environments. Unlike conventional methods that model each agent individually with separate networks, our approach, the Bottom Up Network (BUN), adopts a unique perspective. BUN treats the collective of multi-agents as a unified entity while employing a specialized weight initialization strategy that promotes independent learning. Furthermore, we dynamically establish connections among agents using gradient information, enabling coordination when necessary while maintaining these connections as limited and sparse to effectively manage the computational budget. Our extensive empirical evaluations across a variety of cooperative multi-agent scenarios, including tasks such as cooperative navigation and traffic control, consistently demonstrate BUN's superiority over baseline methods with substantially reduced computational costs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑