arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.02890cs.MAcs.AIcs.LG
收藏

用于高效深度多智能体强化学习的表示学习

Representation Learning For Efficient Deep Multi-Agent Reinforcement Learning

UC Davis(加州大学戴维斯分校);University of South Florida(南佛罗里达大学)

Dom Huh, Prasant Mohapatra

AI总结:

本文提出MAPO-LSO框架,通过引入转移动力学重构和自预测学习的多智能体扩展来构建潜在状态优化方案,有效提升了多智能体强化学习的样本效率和学习性能。

摘要:

样本效率仍是多智能体强化学习(MARL)中的关键挑战。一种有前景的方法是在MARL目标之外,通过辅助学习目标学习有意义的潜在表示空间,以辅助学习成功的控制策略。在我们的工作中,我们提出了MAPO-LSO(基于潜在空间优化的多智能体策略优化),它应用了一种旨在补充MARL训练的综合表示学习方法。具体而言,MAPO-LSO提出了转移动力学重构和自预测学习的多智能体扩展,构建了一种潜在状态优化方案,该方案可以轻松扩展到当前最先进的MARL算法中。实验结果表明,在无需任何额外MARL超参数调优的情况下,与原始的MARL对应方法相比,MAPO-LSO在多样化的MARL任务套件中展现出样本效率和学习性能的显著提升。

英文摘要:

Sample efficiency remains a key challenge in multi-agent reinforcement learning (MARL). A promising approach is to learn a meaningful latent representation space through auxiliary learning objectives alongside the MARL objective to aid in learning a successful control policy. In our work, we present MAPO-LSO (Multi-Agent Policy Optimization with Latent Space Optimization) which applies a form of comprehensive representation learning devised to supplement MARL training. Specifically, MAPO-LSO proposes a multi-agent extension of transition dynamics reconstruction and self-predictive learning that constructs a latent state optimization scheme that can be trivially extended to current state-of-the-art MARL algorithms. Empirical results demonstrate MAPO-LSO to show notable improvements in sample efficiency and learning performance compared to its vanilla MARL counterpart without any additional MARL hyperparameter tuning on a diverse suite of MARL tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑