Aurora: 一种基于工具使用的统一视频编辑框架
Aurora: Unified Video Editing with a Tool-Using Agent
AI总结：
本文提出Aurora框架，通过结合增强的视觉语言模型（VLM）代理和统一视频扩散变换器，解决视频编辑中的文本和视觉不充分问题，提升了视频编辑的灵活性和准确性。
摘要：
近期的视频编辑模型趋于统一的条件设计：一个扩散变换器同时消耗文本、源视频和参考图像，并且一组权重可以用于替换、删除、风格迁移和参考驱动的插入。该设计具有灵活性，但假设用户已经提供了模型准备的文本、参考图像和局部编辑的空间定位，而实际需求往往省略这些。我们提出了Aurora，一种基于代理的视频编辑框架，将增强的视觉语言模型（VLM）代理与统一视频扩散变换器相结合。VLM代理将原始用户请求映射到与变换器条件通道对齐的结构化编辑计划，从而在生成前解决文本和视觉不充分问题。我们使用监督数据训练VLM代理进行完整的编辑计划和参考图像选择，同时结合偏好对进行鲁棒的工具使用和指令细化。我们引入AgentEdit-Bench来评估在文本和视觉不充分情况下的代理增强视频编辑。在AgentEdit-Bench和两个现有视频编辑基准测试中，实验表明Aurora优于仅基于指令的基线，并且VLM代理可以转移到兼容的冻结视频编辑模型中。项目页面：https://yeates.github.io/Aurora-Page
英文摘要：
Recent video editing models have converged on a unified conditioning design: a single diffusion transformer jointly consumes text, source video, and reference images, and one set of weights covers replacement, removal, style transfer, and reference-driven insertion. The design is flexible, but it assumes that the user already provides model-ready text, reference images, and spatial grounding for local edits, which real requests often omit. We present Aurora, an agentic video editing framework that pairs a tool-augmented vision-language model (VLM) agent with a unified video diffusion transformer. The VLM agent maps a raw user request to a structured edit plan aligned with the transformer's conditioning channels, thereby resolving textual and visual underspecification before generation. We train the VLM agent with supervised data for complete edit planning and reference-image selection, together with preference pairs for robust tool use and instruction refinement. We introduce AgentEdit-Bench to evaluate agent-enhanced video editing under textual and visual underspecification. Experiments on AgentEdit-Bench and two existing video editing benchmarks show that Aurora improves over instruction-only baselines and that the VLM agent transfers to compatible frozen video editing models. Project page: https://yeates.github.io/Aurora-Page
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
