arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.02064cs.AI
收藏

E-MAPP:高效的并行程序引导多智能体强化学习

E-MAPP: Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidance

Stanford University(斯坦福大学);Mila(米拉人工智能研究所);Université de Montréal(蒙特利尔大学);Tsinghua University(清华大学);Shanghai Qi Zhi Institute(上海期智研究院);Southeast University(东南大学)

Can Chang, Ni Mu, Jiajun Wu, Ling Pan, Huazhe Xu

摘要:

多智能体强化学习(MARL)的一个关键挑战是多个智能体有效地完成复杂的长期任务。智能体在共同目标上的合作、复杂任务的划分以及通过几个阶段的规划来取得进展方面往往存在困难。我们建议解决这些挑战,指导代理与程序设计的并行化,因为程序作为一种表示包含丰富的结构和语义信息,并广泛用作长期任务的抽象。具体来说,我们引入了高效的多智能体强化学习与并行程序指导(E-MAPP),一个新的框架,利用并行程序来指导多个代理有效地完成目标,需要规划的$10+$阶段。E-MAPP集成了并行程序的结构信息,促进了基于程序语义的协作行为,并通过任务分配器提高了时间效率。我们进行了广泛的实验上的一系列具有挑战性的,长视野的合作任务在过度烹饪的环境。结果表明,E-MAPP在完成率、时间效率和零次泛化能力(zero-shot generalization ability)方面大幅优于强基线。

英文摘要:

A critical challenge in multi-agent reinforcement learning(MARL) is for multiple agents to efficiently accomplish complex, long-horizon tasks. The agents often have difficulties in cooperating on common goals, dividing complex tasks, and planning through several stages to make progress. We propose to address these challenges by guiding agents with programs designed for parallelization, since programs as a representation contain rich structural and semantic information, and are widely used as abstractions for long-horizon tasks. Specifically, we introduce Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidance(E-MAPP), a novel framework that leverages parallel programs to guide multiple agents to efficiently accomplish goals that require planning over $10+$ stages. E-MAPP integrates the structural information from a parallel program, promotes the cooperative behaviors grounded in program semantics, and improves the time efficiency via a task allocator. We conduct extensive experiments on a series of challenging, long-horizon cooperative tasks in the Overcooked environment. Results show that E-MAPP outperforms strong baselines in terms of the completion rate, time efficiency, and zero-shot generalization ability by a large margin.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑