变分离线多智能体技能发现
Variational Offline Multi-agent Skill Discovery
AI总结：
针对多智能体任务中自动提取子群协调模式的挑战，提出两种自编码器方案VO-MASD-3D和VO-MASD-Hier，通过动态分组函数捕获子群与时间级抽象以形成多智能体技能，在离线多任务数据上实现技能跨任务迁移，显著优于现有分层MARL方法并降低稀疏奖励下的学习难度。
摘要：
技能是为序列决策建立的有效时间抽象，能够为长周期任务实现高效的分层学习，并通过其可迁移性促进多任务学习。尽管已有大量研究，但在多智能体场景中仍存在研究空白，尤其是在多智能体任务中自动提取子群协调模式方面。在这种情况下，我们提出了两种新颖的自编码器方案：VO-MASD-3D 和 VO-MASD-Hier，以同时捕获子群级和时间级的抽象并形成多智能体技能，这首次解决了上述挑战。这些方案的一个核心算法组件是动态分组函数，它能够根据任务中智能体的交互自动检测潜在子群。此外，我们的方法可应用于离线多任务数据，并且发现的子群技能可以在无需重新训练的情况下跨相关任务进行迁移。在 StarCraft 任务上的实证评估表明，我们的方法显著优于现有的分层多智能体强化学习（MARL）方法。此外，使用我们的方法发现的技能能够有效降低在具有延迟和稀疏奖励信号的 MARL 场景中的学习难度。代码库可在 https://github.com/LucasCJYSDL/VOMASD 获取。
英文摘要：
Skills are effective temporal abstractions established for sequential decision making, which enable efficient hierarchical learning for long-horizon tasks and facilitate multi-task learning through their transferability. Despite extensive research, research gaps remain in multi-agent scenarios, particularly for automatically extracting subgroup coordination patterns in a multi-agent task. In this case, we propose two novel auto-encoder schemes: VO-MASD-3D and VO-MASD-Hier, to simultaneously capture subgroup- and temporal-level abstractions and form multi-agent skills, which firstly solves the aforementioned challenge. An essential algorithm component of these schemes is a dynamic grouping function that can automatically detect latent subgroups based on agent interactions in a task. Further, our method can be applied to offline multi-task data, and the discovered subgroup skills can be transferred across relevant tasks without retraining. Empirical evaluations on StarCraft tasks indicate that our approach significantly outperforms existing hierarchical multi-agent reinforcement learning (MARL) methods. Moreover, skills discovered using our method can effectively reduce the learning difficulty in MARL scenarios with delayed and sparse reward signals. The codebase is available at https://github.com/LucasCJYSDL/VOMASD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
