多智能体系统中的熵增强多目标策略优化

中文摘要 AI 辅助

部署在海洋、外星前哨等场景的自主智能体团队，需协同行动以在多个相互冲突的目标间实现最优结果。NSGA-II等多目标进化算法会优化目标空间的多样性，但忽略行为空间的多样性，可能导致过早收敛及行为崩溃，而这些行为本可区分不同外部条件下的策略。为解决该问题，我们提出熵增强的策略评估策略，将熵奖励纳入智能体适应度分数，抑制演化种群中的行为同质性。我们的方法在保留基础帕累托优化框架的同时，通过行为空间多样性信号增强策略评估，旨在鼓励多智能体领域中探索行为各异的策略。我们在具有不同奖励结构的漫游者领域实验中评估了该方法，观察到与NSGA-II基准相比，超体积提升最高达48%，表明行为多样性是改进多目标多智能体进化优化的有前景且未被充分探索的方向。

英文摘要

Autonomous agent teams deployed in settings such as marine and extraterrestrial outposts must coordinate actions to achieve optimal outcomes across multiple competing objectives. Multi-objective evolutionary algorithms such as NSGA-II optimize for diversity in the objective space, but neglect diversity in the behavior space, possibly leading to premature convergence and a collapse in behaviors that may differentiate policies in different external conditions. To address this, we introduce an entropy-augmented policy evaluation strategy that incorporates an entropy bonus into agent fitness scores, discouraging behavioral homogeneity across the evolving population. By augmenting policy evaluation with a behavior-space diversity signal while preserving the underlying Pareto optimization framework, our method is designed to encourage exploration of behaviorally distinct policies in multiagent domains. We evaluate our approach across rover-domain experiments with qualitatively distinct reward structures and observe hypervolume improvements of up to 48% relative to the NSGA-II baseline, suggesting that behavioral diversity is a promising and underexplored direction for improving multi-objective multiagent evolutionary optimization.

发表机构

• The Collaborative Robotics and Intelligent Systems (CoRIS) Institute（协作机器人与智能系统研究所（CoRIS研究所））
• Oregon State University（俄勒冈州立大学）

机构由 AI 辅助整理，请以论文原文为准。