VideoWeaver: 评估与进化智能体长视频生成技能
VideoWeaver: Evaluating and Evolving Skills for Agentic Long Video Generation
AI总结：
提出VideoWeaver框架，让智能体自主组合基础技能生成视频，并设计智能体裁判评估过程与结果，通过技能进化算法提升生成质量。
摘要：
最近的智能体框架如Claude Code、Codex和OpenClaw在工具使用和编排方面表现强劲，但它们能否处理长视频生成这一长时多模态任务仍待探索。与早期手工设计管线的视频智能体不同，这些框架可以构建和优化自己的工作流程。我们提出VideoWeaver，一个评估和进化长视频生成技能的智能体框架和基准测试，其中智能体通过将基础技能组合成自己的工作流程（而非遵循预定义管线）将单个指令转化为长视频。该基准测试包含16个任务类别和285个案例，参考信息涵盖文本、图像、音频、视频及其组合。由于错误可能出现在任何阶段而不仅仅是最终视频，我们提出一种智能体裁判，它检查执行轨迹和最终视频，并将其评分基于元数据和中间文件等证据。利用这一反馈，我们进一步设计了一种技能进化算法，用于优化和合并智能体的技能。在多个框架和模型上，我们发现显式的组合技能比单独使用基础技能更能改善生成过程，技能进化进一步提高了输出质量，并且不同框架和模型选择之间的性能差异显著。所提出的智能体裁判也与人类判断高度一致，尤其是在过程指标上。代码和数据集可在https://github.com/JianhuiWei7/VideoWeaver获取。
英文摘要：
Agentic long video generation requires planning, tool orchestration, and cross-clip coordination over a long horizon. Most existing video agents either rely on static, human-crafted workflows, which require substantial manual effort and poorly adapt across tasks, or iteratively refine the output of the current task without persistently distilling execution experience into reusable skills for future tasks. We introduce VideoWeaver, an agent harness and benchmark that evaluates and evolves skills for long video generation. Given a single high-level instruction, an agent dynamically composes foundation skills into its own workflow rather than following a predefined pipeline. We construct a benchmark of 16 task categories and 285 cases, with references spanning text, image, audio, video, and their combinations. We further propose an evidence-grounded agent-as-judge that inspects both the execution trace and the final video to diagnose process and output failures. Based on this feedback, our evolution algorithm progressively refines category-level composition and creator skills, allowing recurring experience to guide dynamically constructed workflows for unseen cases. Experiments show that explicit composition skills improve the generation process over foundation skills alone, while skill evolution further improves output quality and generalizes to unseen cases. Incorporating judge feedback yields additional gains, especially on output metrics, and the agent-as-judge aligns well with human, particularly on process metrics. Code is available at https://github.com/JianhuiWei7/VideoWeaver.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
