发表机构
School of Computer Science and technology, Tongji University（同济大学计算机科学与技术学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对多智能体强化学习的结构化噪声效应，提出分层协作框架SIGMA，通过分组聚合与组间注意力提升含噪观测下的鲁棒性，且在无噪环境中保持竞争力。
AI 中文摘要
合作多智能体强化学习（MARL）在含噪观测下维持鲁棒协作面临重大挑战。尽管观测干扰通常在智能体间独立引入，但其对协作决策的下游影响可通过底层协作结构呈现为结构化形式。我们将此现象定义为结构化噪声效应：噪声诱导的决策效应在具有更强任务相关依赖的智能体间表现出局部相关性，而在不同智能体及局部结构间保持全局异质性。然而，现有鲁棒MARL方法极少明确表征或利用这种依赖结构的噪声效应。为解决该局限，我们提出SIGMA——一种分层协作框架，利用协作结构在含噪观测下学习鲁棒表征。SIGMA首先通过基于密度的分组将智能体组织为自适应局部结构，执行组内共识聚合以保留共享任务相关信息，同时平滑智能体特定的表征偏差；随后组间注意力机制自适应整合不同组间信息，以维持全局协作，同时适配各组的异质贡献。在《星际争霸II》含噪观测任务上的实验，实证验证了结构化噪声效应，并证明SIGMA在观测噪声下持续提升鲁棒性，同时在无噪声环境中保持有竞争力的性能。
英文摘要
Cooperative multi-agent reinforcement learning (MARL) faces significant challenges in maintaining robust coordination under noisy observations. Although observation disturbances are often introduced independently across agents, their downstream effects on cooperative decision-making can become structured through underlying cooperation structures. We characterize this phenomenon as structured noise effects, where noise-induced decision effects exhibit local correlation among agents with stronger task-related dependencies while remaining globally heterogeneous across different agents and local structures. Existing robust MARL methods, however, rarely explicitly characterize or exploit such structure-dependent noise effects. To address this limitation, we propose SIGMA, a hierarchical collaboration framework that exploits cooperation structures to learn robust representations under noisy observations. SIGMA first organizes agents into adaptive local structures through density-based grouping and performs intra-group consensus aggregation to preserve shared task-relevant information while smoothing agent-specific representation deviations. Inter-group attention then adaptively integrates information across different groups to preserve global coordination while accommodating their heterogeneous contributions. Experiments on noisy-observation tasks in StarCraft II empirically validate the structured noise effects and demonstrate that SIGMA consistently improves robustness under observation noise while maintaining competitive performance in noise-free environments.
