arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.06750cs.MAcs.LGcs.RO
收藏

通过任务无关通信实现多智能体协作的泛化

Generalising Multi-Agent Cooperation through Task-Agnostic Communication

University of Oxford(牛津大学);University of Cambridge(剑桥大学)

Dulhan Jayalath, Steven Morad, Amanda Prorok

AI总结:

提出一种自监督、任务无关的集合自编码器通信策略,可从可变数量智能体观测学习固定潜在 Markov 状态,并在新任务和规模扩展中优于任务特定方法。

摘要:

在协作式多机器人问题中,现有的多智能体强化学习(MARL)通信方法几乎完全是任务特定的,需要为每个独特任务训练新的通信策略。我们通过引入一种适用于给定环境中任意任务的通信策略来解决这一低效问题。我们在没有任务特定奖励指导的情况下,以自监督方式使用集合自编码器预训练该通信策略。我们的目标是从数量可变的智能体观测中学习固定大小的潜在 Markov 状态。在温和假设下,我们证明使用我们潜在表示的策略保证收敛,并给出由我们的 Markov 状态近似所引入值误差的上界。我们的方法能够无缝适应新任务,而无需微调通信策略,可良好支持扩展到比训练期间更多的智能体,并能检测环境中的分布外事件。在多种 MARL 场景上的实验结果验证了我们方法的有效性,其在未见过的任务中优于任务特定通信策略。本工作的实现可在 https://github.com/proroklab/task-agnostic-comms 获取。

英文摘要:

Existing communication methods for multi-agent reinforcement learning (MARL) in cooperative multi-robot problems are almost exclusively task-specific, training new communication strategies for each unique task. We address this inefficiency by introducing a communication strategy applicable to any task within a given environment. We pre-train the communication strategy without task-specific reward guidance in a self-supervised manner using a set autoencoder. Our objective is to learn a fixed-size latent Markov state from a variable number of agent observations. Under mild assumptions, we prove that policies using our latent representations are guaranteed to converge, and upper bound the value error introduced by our Markov state approximation. Our method enables seamless adaptation to novel tasks without fine-tuning the communication strategy, gracefully supports scaling to more agents than present during training, and detects out-of-distribution events in an environment. Empirical results on diverse MARL scenarios validate the effectiveness of our approach, surpassing task-specific communication strategies in unseen tasks. Our implementation of this work is available at https://github.com/proroklab/task-agnostic-comms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑