面向演化多智能体系统的战略协调：一种分层强化与集体学习方法
Optimization without Future Compromises? Decentralized Coordination via Collective and Reinforcement Learning
AI总结：
本文提出分层强化与集体学习（HRCL）方法，通过高层MARL策略减少动作空间并约束帕累托最优，低层集体学习实现低通信去中心化协调，在智慧城市等场景中显著提升性能、可扩展性与适应性。
摘要：
演化多智能体系统中的去中心化组合优化面临重大挑战，要求智能体在平衡长期决策与短期优化集体结果的同时，在不可预见的变化下保持交互智能体的自主性。强化学习提供了一种通过动态规划对顺序决策进行建模以预测未来环境变化的方式。然而，将多智能体强化学习（MARL）应用于去中心化组合优化问题仍然是一个开放性挑战，原因在于联合状态-动作空间的指数增长、高昂的通信开销以及集中式训练中的隐私问题。为了解决这些局限性，本文提出了分层强化与集体学习（HRCL），这是一种基于分层框架同时利用MARL和去中心化集体学习的新方法。智能体使用MARL采取高层策略，对可能的计划进行分组以减少动作空间，并约束智能体行为以实现帕累托最优。同时，低层集体学习层以最少的通信确保智能体之间高效且去中心化的协调决策。在合成场景和现实世界智慧城市应用模型（包括能源自我管理和无人机群感知）中进行的大量实验表明，与单独的MARL和集体学习方法相比，HRCL显著提升了性能、可扩展性和适应性，实现了双赢的综合解决方案。
英文摘要：
Efficient resource allocation in multi-agent systems requires autonomous agents to coordinate their decisions while balancing system-wide objectives with individual costs. This becomes increasingly challenging over long time horizons, where decisions that improve the current allocation may compromise future resource allocation, while decentralized agents have limited observations of the overall system. Multi-agent reinforcement learning (MARL) can learn such long-term dependencies via local observations, but directly applying it to large-scale coordination leads to rapidly growing decision spaces and inefficient training. To this end, we propose Hierarchical Reinforcement and Collective Learning (HRCL), a hierarchical framework that uses MARL to guide, rather than replace, decentralized multi-agent coordination. At the high level, MARL learns strategies that restrict the alternatives considered during coordination and guide agents in balancing system-wide and individual objectives. At the low level, agents perform efficient decentralized coordination under this strategic guidance. This separation reduces the learning space and allows short-term coordination trade-offs to be evaluated according to their long-term effects. Experiments on a synthetic benchmark show that HRCL converges substantially faster than standalone MARL and reduces system-wide and individual costs by 35.53% and 27.05%, respectively. Evaluations on energy self-management and drone swarm sensing further show improved resource allocation, power-peak regulation, and sensing efficiency. These results show that learning strategic guidance for an existing coordination process can retain scalable decentralized coordination without letting short-term decisions compromise future resource allocation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
