从离线多任务数据中学习可泛化的多智能体协作技能
Learning Generalizable Skills from Offline Multi-Task Data for Multi-Agent Cooperation
AI总结：
针对离线多任务多智能体强化学习中技能学习缺乏协作时序知识和任务特定技能的问题，提出HiSSD层级框架联合学习共同与任务特定技能，在MuJoCo和SMAC基准上验证了其在未见任务上的优越泛化性能。
摘要：
从离线多任务数据中学习能够泛化到具有不同智能体和目标数量的未见任务的多智能体协作策略，在许多场景中都是一个有吸引力的研究问题。尽管将多个任务中的通用行为模式聚合为技能以提升策略迁移是一种有前景的方法，但两个主要挑战阻碍了离线多任务多智能体强化学习中技能学习的进一步发展。首先，从各种动作序列中提取通用协作行为作为共同技能，缺乏将协作时序知识融入其中。其次，现有工作仅涉及共同技能，无法在每个任务中自适应地选择独立知识作为任务特定技能，以实现细粒度的动作执行。为了应对这些挑战，我们提出了层级化与分离式技能发现（HiSSD），一种通过技能学习实现可泛化离线多任务多智能体强化学习的新方法。HiSSD利用一个层级框架，联合学习共同技能和任务特定技能。共同技能学习协作时序知识，并支持离线多任务多智能体强化学习中的样本内利用。任务特定技能表示每个任务的先验知识，并实现任务引导的细粒度动作执行。为了验证我们方法的先进性，我们在多智能体MuJoCo和SMAC基准上进行了实验。在使用HiSSD在离线多任务数据上训练策略后，实证结果表明，HiSSD能够分配有效的协作行为，并在未见任务中获得优越的性能。
英文摘要：
Learning cooperative multi-agent policy from offline multi-task data that can generalize to unseen tasks with varying numbers of agents and targets is an attractive problem in many scenarios. Although aggregating general behavior patterns among multiple tasks as skills to improve policy transfer is a promising approach, two primary challenges hinder the further advancement of skill learning in offline multi-task MARL. Firstly, extracting general cooperative behaviors from various action sequences as common skills lacks bringing cooperative temporal knowledge into them. Secondly, existing works only involve common skills and can not adaptively choose independent knowledge as task-specific skills in each task for fine-grained action execution. To tackle these challenges, we propose Hierarchical and Separate Skill Discovery (HiSSD), a novel approach for generalizable offline multi-task MARL through skill learning. HiSSD leverages a hierarchical framework that jointly learns common and task-specific skills. The common skills learn cooperative temporal knowledge and enable in-sample exploitation for offline multi-task MARL. The task-specific skills represent the priors of each task and achieve a task-guided fine-grained action execution. To verify the advancement of our method, we conduct experiments on multi-agent MuJoCo and SMAC benchmarks. After training the policy using HiSSD on offline multi-task data, the empirical results show that HiSSD assigns effective cooperative behaviors and obtains superior performance in unseen tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
