GenMAC：基于多智能体协作的组合式文本到视频生成
GenMAC: Compositional Text-to-Video Generation with Multi-Agent Collaboration
AI总结：
本文提出多智能体协作框架 GenMAC，通过设计、生成与迭代重新设计流程及自路由修正机制，提升复杂组合文本提示下的视频生成质量。
摘要：
近年来，文本到视频生成模型取得了显著进展。然而，它们仍难以根据组合式文本提示生成复杂动态场景，例如多个对象的属性绑定、与不同对象相关的时间动态，以及对象之间的交互。我们的核心动机是，复杂任务可以被分解为更简单的任务，每个任务由一个角色专门化的 MLLM 智能体处理。多个智能体可以协同工作，以实现面向复杂目标的集体智能。我们提出了 GenMAC，这是一个支持组合式文本到视频生成的迭代式多智能体框架。协作工作流包括三个阶段：设计、生成和重新设计，并在生成与重新设计阶段之间设置迭代循环，以逐步验证和细化生成的视频。重新设计阶段是最具挑战性的阶段，旨在验证生成的视频、提出修正建议，并为下一轮生成重新设计文本提示、逐帧布局和引导尺度。为避免单个 MLLM 智能体产生幻觉，我们将该阶段分解为四个顺序执行的基于 MLLM 的智能体：验证智能体、建议智能体、修正智能体和输出结构化智能体。此外，为应对组合式文本到视频生成的多种场景，我们设计了一种自路由机制，从一组各自专用于一种场景的修正智能体中自适应地选择合适的修正智能体。大量实验证明了 GenMAC 的有效性，其在组合式文本到视频生成中达到了最先进性能。
英文摘要：
Text-to-video generation models have shown significant progress in the recent years. However, they still struggle with generating complex dynamic scenes based on compositional text prompts, such as attribute binding for multiple objects, temporal dynamics associated with different objects, and interactions between objects. Our key motivation is that complex tasks can be decomposed into simpler ones, each handled by a role-specialized MLLM agent. Multiple agents can collaborate together to achieve collective intelligence for complex goals. We propose GenMAC, an iterative, multi-agent framework that enables compositional text-to-video generation. The collaborative workflow includes three stages: Design, Generation, and Redesign, with an iterative loop between the Generation and Redesign stages to progressively verify and refine the generated videos. The Redesign stage is the most challenging stage that aims to verify the generated videos, suggest corrections, and redesign the text prompts, frame-wise layouts, and guidance scales for the next iteration of generation. To avoid hallucination of a single MLLM agent, we decompose this stage to four sequentially-executed MLLM-based agents: verification agent, suggestion agent, correction agent, and output structuring agent. Furthermore, to tackle diverse scenarios of compositional text-to-video generation, we design a self-routing mechanism to adaptively select the proper correction agent from a collection of correction agents each specialized for one scenario. Extensive experiments demonstrate the effectiveness of GenMAC, achieving state-of-the art performance in compositional text-to-video generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
