arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.00730cs.AIcs.LGcs.MA
收藏

MO-MIX:基于深度强化学习的多目标多智能体协作决策制定

MO-MIX: Multi-Objective Multi-Agent Cooperative Decision-Making With Deep Reinforcement Learning

School of Automation, Central South University(中南大学自动化学院);Texas A&M University at Qatar(卡塔尔大学塔拉斯阿姆大学)

Tianmeng Hu, Biao Luo, Chunhua Yang, Tingwen Huang

AI总结:

MO-MIX通过集中训练与分散执行框架,解决多目标多智能体协作决策问题,提升非支配解的均匀性并降低计算成本。

摘要:

深度强化学习(RL)已被广泛应用于解决复杂决策问题。在许多现实世界场景中,任务通常具有多个冲突的目标,并可能需要多个智能体协作,这构成了多目标多智能体决策问题。然而,仅少数工作涉及这一交叉领域。现有方法局限于单独领域,只能处理具有单一目标的多智能体决策或具有单一智能体的多目标决策。在本文中,我们提出了MO-MIX来解决多目标多智能体强化学习(MOMARL)问题。我们的方法基于集中训练与分散执行(CTDE)框架。一个表示对目标偏好的权重向量被输入到分散智能体网络中,作为局部动作值函数估计的条件,同时使用具有并行架构的混合网络来估计联合动作值函数。此外,应用了探索引导方法以提高最终非支配解的均匀性。实验表明,所提出的方法能够有效解决多目标多智能体协作决策问题,并生成帕累托集的近似值。我们的方法不仅在所有四种评估指标上显著优于基线方法,而且计算成本更低。

英文摘要:

Deep reinforcement learning (RL) has been applied extensively to solve complex decision-making problems. In many real-world scenarios, tasks often have several conflicting objectives and may require multiple agents to cooperate, which are the multi-objective multi-agent decision-making problems. However, only few works have been conducted on this intersection. Existing approaches are limited to separate fields and can only handle multi-agent decision-making with a single objective, or multi-objective decision-making with a single agent. In this paper, we propose MO-MIX to solve the multi-objective multi-agent reinforcement learning (MOMARL) problem. Our approach is based on the centralized training with decentralized execution (CTDE) framework. A weight vector representing preference over the objectives is fed into the decentralized agent network as a condition for local action-value function estimation, while a mixing network with parallel architecture is used to estimate the joint action-value function. In addition, an exploration guide approach is applied to improve the uniformity of the final non-dominated solutions. Experiments demonstrate that the proposed method can effectively solve the multi-objective multi-agent cooperative decision-making problem and generate an approximation of the Pareto set. Our approach not only significantly outperforms the baseline method in all four kinds of evaluation metrics, but also requires less computational cost.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑