arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.00741cs.AIcs.LGcs.RO
收藏

带安全约束的离线多智能体强化学习扩散模型

Diffusion Models for Offline Multi-agent Reinforcement Learning with Safety Constraints

Jianuo Huang

AI总结:

本文提出一种基于CTDE架构并集成扩散模型的离线多智能体强化学习框架,通过轨迹预测生成和专门安全算法缓解风险,在DSRL基准上验证其能同时满足严格安全约束并取得优于现有方法的性能。

摘要:

在多智能体强化学习(MARL)的最新进展中,其应用已扩展到各种安全关键场景。然而,大多数方法侧重于在线学习,这在部署到现实环境中时会带来巨大风险。为应对这一挑战,我们提出了一种将扩散模型集成到MARL范式中的创新框架。该方法通过风险缓解显著增强了多个智能体所采取动作的安全性,同时对协调动作进行建模。我们的框架基于集中训练与分散执行(CTDE)架构,并通过扩散模型进行预测轨迹生成。此外,我们引入了一种专门算法以进一步确保运行安全。我们在DSRL基准上对照基线评估了我们的模型。实验结果表明,我们的模型不仅遵守严格的安全约束,而且与现有方法相比取得了更优越的性能。这凸显了我们的方法在推进MARL在现实应用中安全性与有效性方面的潜力。

英文摘要:

In recent advancements in Multi-agent Reinforcement Learning (MARL), its application has extended to various safety-critical scenarios. However, most methods focus on online learning, which presents substantial risks when deployed in real-world settings. Addressing this challenge, we introduce an innovative framework integrating diffusion models within the MARL paradigm. This approach notably enhances the safety of actions taken by multiple agents through risk mitigation while modeling coordinated action. Our framework is grounded in the Centralized Training with Decentralized Execution (CTDE) architecture, augmented by a Diffusion Model for prediction trajectory generation. Additionally, we incorporate a specialized algorithm to further ensure operational safety. We evaluate our model against baselines on the DSRL benchmark. Experiment results demonstrate that our model not only adheres to stringent safety constraints but also achieves superior performance compared to existing methodologies. This underscores the potential of our approach in advancing the safety and efficacy of MARL in real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑