协同扩散：无需多智能体演示生成多智能体行为
Coordinated Diffusion: Generating Multi-Agent Behavior Without Multi-Agent Demonstrations
AI总结：
本文提出Coordinated Diffusion框架，通过用户定义的多智能体成本函数将独立训练的单智能体扩散策略耦合，无需多智能体演示数据，实现多智能体行为协调。
摘要：
基于生成模型的模仿学习在建模复杂单智能体行为方面已证明有效。然而，通过模仿学习教授多智能体系统（如多个臂或车辆）协调行为受限于根本的数据瓶颈：随着智能体数量增加，联合状态-动作空间呈指数增长，收集足够的协调多智能体演示变得极其昂贵。本文提出：如何利用单智能体演示数据学习多智能体策略？我们提出了协同扩散（CoDi），一种框架，通过用户定义的多智能体成本函数将独立训练的单智能体扩散策略耦合，无需任何协调演示。我们推导出一种新的基于扩散的采样方案，其中扩散分数函数分解为独立的单智能体预训练基础策略加上一个成本驱动的指导项，该项将这些基础策略协调成连贯的多智能体行为。我们显示，该指导项可通过梯度自由方式估计，使CoDi适用于黑盒、非可微分成本函数，无需额外训练。理论上和实验上，我们分析了这种组合在何种条件下能忠实近似目标多智能体行为。我们发现演示数据与成本函数有互补作用：单智能体演示必须覆盖期望的多智能体行为支持，而成本函数必须从这些单智能体策略的乘积中促进期望行为。在仿真和硬件实验中的双臂操作任务中，我们发现CoDi能从单智能体数据中发现稳健的协调行为，比多智能体基线更数据高效，并突显了联合指导、基础策略支持和成本设计的重要性。
英文摘要：
Imitation learning powered by generative models has proven effective for modeling complex single-agent behaviors. However, teaching multi-agent systems, like multiple arms or vehicles, to coordinate through imitation learning is hindered by a fundamental data bottleneck: as the joint state-action space grows exponentially with the number of agents, collecting a sufficient amount of coordinated multi-agent demonstrations becomes extremely costly. In this work, we ask: how can we leverage single-agent demonstration data to learn multi-agent policies? We present Coordinated Diffusion (CoDi), a framework that couples independently trained single-agent diffusion policies through a user-defined multi-agent cost function, without requiring any coordinated demonstrations. We derive a new diffusion-based sampling scheme wherein the diffusion score function decomposes into independent, single-agent pre-trained base policies plus a cost-driven guidance term that coordinates these base policies into cohesive multi-agent behavior. We show that this guidance term can be estimated in a gradient-free manner, making CoDi applicable to black-box, non-differentiable cost functions without additional training. Theoretically and empirically, we analyze the conditions under which this composition can faithfully approximate a target multi-agent behavior. We find a complementary role for demonstration data versus the cost function: single-agent demonstrations must cover the support of the desired multi-agent behavior, while the cost function must promote desired behavior from this product of single-agent policies. Our results in simulation and hardware experiments of a two-arm manipulation task show that CoDi discovers robust coordinated behavior from single-agent data, is more data-efficient than multi-agent baselines, and highlights the importance of joint guidance, base policy support, and cost design.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
