面向多智能体强化学习中增强多任务泛化的混合训练
Hybrid Training for Enhanced Multi-task Generalization in Multi-agent Reinforcement Learning
AI总结：
本文提出HyGen混合MARL框架，通过整合离线技能提取与在线策略训练，在CTDE范式下利用混合回放缓冲区实现高效的多任务泛化，并在星际争霸多智能体挑战赛中优于现有纯在线和纯离线方法。
摘要：
在多智能体强化学习（MARL）中，实现对多样化智能体和目标的多任务泛化面临重大挑战。现有的在线MARL算法主要关注单任务性能，但其缺乏多任务泛化能力通常会导致大量计算浪费和有限的实际应用价值。同时，现有的离线多任务MARL方法严重依赖数据质量，往往在未见任务上表现不佳。在本文中，我们提出了HyGen，一种新颖的混合MARL框架，即面向增强多任务泛化的混合训练，该框架整合了在线学习和离线学习，以确保多任务泛化和训练效率。具体而言，我们的框架从离线多任务数据集中提取潜在的通用技能。然后，我们在集中训练与分散执行（CTDE）范式下训练策略以选择最优技能。在此阶段，我们利用一个同时整合离线数据和在线交互的回放缓冲区。我们通过实验证明，我们的框架能够有效提取和精炼通用技能，从而对未见任务产生令人印象深刻的泛化能力。在星际争霸多智能体挑战赛上的对比分析表明，HyGen优于多种现有的纯在线和纯离线方法。
英文摘要：
In multi-agent reinforcement learning (MARL), achieving multi-task generalization to diverse agents and objectives presents significant challenges. Existing online MARL algorithms primarily focus on single-task performance, but their lack of multi-task generalization capabilities typically results in substantial computational waste and limited real-life applicability. Meanwhile, existing offline multi-task MARL approaches are heavily dependent on data quality, often resulting in poor performance on unseen tasks. In this paper, we introduce HyGen, a novel hybrid MARL framework, Hybrid Training for Enhanced Multi-Task Generalization, which integrates online and offline learning to ensure both multi-task generalization and training efficiency. Specifically, our framework extracts potential general skills from offline multi-task datasets. We then train policies to select the optimal skills under the centralized training and decentralized execution paradigm (CTDE). During this stage, we utilize a replay buffer that integrates both offline data and online interactions. We empirically demonstrate that our framework effectively extracts and refines general skills, yielding impressive generalization to unseen tasks. Comparative analyses on the StarCraft multi-agent challenge show that HyGen outperforms a wide range of existing solely online and offline methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
