异构示范中团队行为的层次化模仿学习
Hierarchical Imitation Learning of Team Behavior from Heterogeneous Demonstrations
AI总结：
本文提出层次化模仿学习算法DTIL，从异构示范中学习多模态团队行为，通过层次化策略和分布匹配方法，有效处理复杂序列任务，优于现有MAIL基线。
摘要：
成功的协作要求团队成员保持对齐，尤其是在复杂的序列任务中。团队成员必须动态协调执行哪些子任务以及以何种顺序执行。然而，现实世界的约束，如部分可观测性和有限的通信带宽，往往导致次优的协作。即使在专家团队中，同一任务也可以以多种方式执行。为了开发针对此类任务的多智能体系统和人类-人工智能团队，我们对多模态团队行为的数据驱动学习感兴趣。多智能体模仿学习（MAIL）为从示范中数据驱动地学习团队行为提供了一个有前景的框架，但现有方法在处理异构示范时存在困难，因为它们假设所有示范都源自单一团队策略。因此，在这项工作中，我们引入了DTIL：一种层次化MAIL算法，旨在学习复杂序列任务中的多模态团队行为。DTIL用层次化策略表示每个团队成员，并以分解的方式从异构团队示范中学习这些策略。通过采用分布匹配方法，DTIL减轻了复合误差，并能有效扩展到长视野和连续状态表示。实验结果表明，DTIL优于MAIL基线，并在各种协作场景中准确建模了团队行为。
英文摘要：
Successful collaboration requires team members to stay aligned, especially in complex sequential tasks. Team members must dynamically coordinate which subtasks to perform and in what order. However, real-world constraints like partial observability and limited communication bandwidth often lead to suboptimal collaboration. Even among expert teams, the same task can be executed in multiple ways. To develop multi-agent systems and human-AI teams for such tasks, we are interested in data-driven learning of multimodal team behaviors. Multi-Agent Imitation Learning (MAIL) provides a promising framework for data-driven learning of team behavior from demonstrations, but existing methods struggle with heterogeneous demonstrations, as they assume that all demonstrations originate from a single team policy. Hence, in this work, we introduce DTIL: a hierarchical MAIL algorithm designed to learn multimodal team behaviors in complex sequential tasks. DTIL represents each team member with a hierarchical policy and learns these policies from heterogeneous team demonstrations in a factored manner. By employing a distribution-matching approach, DTIL mitigates compounding errors and scales effectively to long horizons and continuous state representations. Experimental results show that DTIL outperforms MAIL baselines and accurately models team behavior across a variety of collaborative scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
