arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.06142cs.AI
收藏

MALinZero:面向复杂多智能体规划的高效低维搜索方法

MALinZero: Efficient Low-Dimensional Search for Mastering Complex Multi-Agent Planning

The George Washington University(乔治·华盛顿大学);Carnegie Mellon University(卡内基梅隆大学)

Sizhe Tang, Jiayu Chen, Tian Lan

AI总结:

MALinZero通过将联合动作回报投影到低维上下文线性赌博机空间并结合LinUCT,解决了多智能体MCTS因组合动作空间指数增长导致的搜索低效问题,在多个多智能体基准上取得最优性能。

摘要:

蒙特卡洛树搜索(MCTS)利用树的上置信界(UCT)通过随机采样来平衡探索与利用,在求解复杂规划问题中发挥着重要作用。然而,在多智能体规划中,MCTS面临巨大的组合动作空间,该空间通常随智能体数量呈指数增长。其结果是,树扩展过程中MCTS的分支因子也呈指数增长,使得在树搜索过程中难以高效地进行探索和利用。为此,我们提出MALinZero,这是一种利用联合动作回报的低维表示结构、在复杂多智能体规划中实现高效MCTS的新方法。我们的解决方案可以看作是将联合动作回报投影到可使用上下文线性赌博机问题形式表示的低维空间中。我们使用凸且μ-光滑的损失函数求解该上下文线性赌博机问题,以便对更优的联合动作赋予更高重要性,并缓解潜在的表示局限性;同时推导出一种应用于树的线性上置信界(LinUCT),从而在低维空间中实现新型的多智能体探索与利用。我们分析了MALinZero在低维奖励函数下的遗憾,并通过最大化子模目标提出了一个(1-1/e)近似的联合动作选择算法。MALinZero在矩阵博弈、SMAC和SMACv2等多智能体基准上展示了最先进的性能,在学习速度和最终表现上均优于基于模型和无模型的多智能体强化学习基线。

英文摘要:

Monte Carlo Tree Search (MCTS), which leverages Upper Confidence Bound for Trees (UCTs) to balance exploration and exploitation through randomized sampling, is instrumental to solving complex planning problems. However, for multi-agent planning, MCTS is confronted with a large combinatorial action space that often grows exponentially with the number of agents. As a result, the branching factor of MCTS during tree expansion also increases exponentially, making it very difficult to efficiently explore and exploit during tree search. To this end, we propose MALinZero, a new approach to leverage low-dimensional representational structures on joint-action returns and enable efficient MCTS in complex multi-agent planning. Our solution can be viewed as projecting the joint-action returns into the low-dimensional space representable using a contextual linear bandit problem formulation. We solve the contextual linear bandit problem with convex and $μ$-smooth loss functions -- in order to place more importance on better joint actions and mitigate potential representational limitations -- and derive a linear Upper Confidence Bound applied to trees (LinUCT) to enable novel multi-agent exploration and exploitation in the low-dimensional space. We analyze the regret of MALinZero for low-dimensional reward functions and propose an $(1-\tfrac1e)$-approximation algorithm for the joint action selection by maximizing a sub-modular objective. MALinZero demonstrates state-of-the-art performance on multi-agent benchmarks such as matrix games, SMAC, and SMACv2, outperforming both model-based and model-free multi-agent reinforcement learning baselines with faster learning speed and better performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑