通过动态有向图通信弥合多智能体协同系统中的训练与执行差距
Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems
AI总结:
提出TGCNet算法,通过动态有向图通信和Transformer解码器,在部分可观测任务中实现高效协作,性能优于现有MARL算法。
摘要:
多智能体系统在部分可观测任务中必须学习通信并理解智能体之间的交互,以实现合作目标。然而,现有方法缺乏动态有向通信机制,并依赖全局状态,从而削弱了通信在集中式训练中的作用。为此,我们提出了基于Transformer的图粗化网络(TGCNet),一种新颖的多智能体强化学习(MARL)算法。TGCNet学习动态有向图的拓扑结构以表示通信策略,并集成图粗化网络在训练期间近似全局状态的表示。它还在执行期间利用Transformer解码器进行特征提取。在多个合作MARL基准上的实验表明,与流行的MARL算法相比,TGCNet达到了最先进的性能。进一步的消融研究验证了我们动态有向图通信机制和图粗化网络的有效性。
英文摘要:
Multi-agent systems must learn to communicate and understand interactions between agents to achieve cooperative goals in partially observed tasks. However, existing approaches lack a dynamic directed communication mechanism and rely on global states, thus diminishing the role of communication in centralized training. Thus, we propose the Transformer-based graph coarsening network (TGCNet), a novel multi-agent reinforcement learning (MARL) algorithm. TGCNet learns the topological structure of a dynamic directed graph to represent the communication policy and integrates graph coarsening networks to approximate the representation of global state during training. It also utilizes the Transformer decoder for feature extraction during execution. Experiments on multiple cooperative MARL benchmarks demonstrate state-of-the-art performance compared to popular MARL algorithms. Further ablation studies validate the effectiveness of our dynamic directed graph communication mechanism and graph coarsening networks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。