DIRECT：通过分层多智能体规划和意图引导编辑实现视频混剪创作
DIRECT: Video Mashup Creation via Hierarchical Multi-Agent Planning and Intent-Guided Editing
AI总结：
本文提出DIRECT框架，通过分层多智能体规划和意图引导编辑解决视频混剪中多模态协调问题，提出Mashup-Bench基准并验证了方法在客观和主观评估上的优越性。
摘要：
视频混剪创作代表一种复杂的视频编辑范式，通过重组现有素材来制作引人入胜的视听体验，要求在语义、视觉和听觉维度及多个层面进行复杂的协调。然而，现有的自动化编辑框架往往忽略了跨层面的多模态协调以实现专业级的流畅性，导致不连贯的序列和突兀的视觉过渡及音乐错位。为了解决这个问题，我们将视频混剪创作建模为多模态一致性满足问题（MMCSP），并提出DIRECT框架。模拟专业生产流程，我们的分层多智能体框架将挑战分解为三个级联层次：Screenwriter用于源感知的全局结构锚定，Director用于实例化自适应编辑意图和指导，Editor用于意图引导的镜头序列编辑并进行细粒度优化。我们进一步引入Mashup-Bench，一个包含定制指标的综合基准，用于评估视觉连续性和听觉对齐。大量实验表明，DIRECT在客观指标和人类主观评估上均显著优于最先进的基线方法。项目页面和代码：https://github.com/AK-DREAM/DIRECT
英文摘要：
Video mashup creation represents a complex video editing paradigm that recomposes existing footage to craft engaging audio-visual experiences, demanding intricate orchestration across semantic, visual, and auditory dimensions and multiple levels. However, existing automated editing frameworks often overlook the cross-level multimodal orchestration to achieve professional-grade fluidity, resulting in disjointed sequences with abrupt visual transitions and musical misalignment. To address this, we formulate video mashup creation as a Multimodal Coherency Satisfaction Problem (MMCSP) and propose the DIRECT framework. Simulating a professional production pipeline, our hierarchical multi-agent framework decomposes the challenge into three cascade levels: the Screenwriter for source-aware global structural anchoring, the Director for instantiating adaptive editing intent and guidance, and the Editor for intent-guided shot sequence editing with fine-grained optimization. We further introduce Mashup-Bench, a comprehensive benchmark with tailored metrics for visual continuity and auditory alignment. Extensive experiments demonstrate that DIRECT significantly outperforms state-of-the-art baselines in both objective metrics and human subjective evaluation. Project page and code: https://github.com/AK-DREAM/DIRECT
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
