CoVEBench: 视频编辑模型能处理复杂指令吗?
CoVEBench: Can Video Editing Models Handle Complex Instructions?
AI总结:
提出CoVEBench基准,包含416个源视频和626条多点编辑指令,通过MLLM评估指令遵循度和保真度,揭示当前模型在组合编辑中常遗漏编辑或破坏保留约束。
摘要:
虽然近期基于文本引导的视频编辑模型在基础任务(如风格迁移、物体插入)上表现出色,但现实用户请求具有高度组合性。单个提示通常要求多个耦合编辑,例如同时修改主体、动作和相机视角,同时严格保留无关的时空内容。现有基准受限于孤立编辑和粗粒度全局指标,无法诊断模型如何处理此类复杂工作流。为弥补这一空白,我们引入CoVEBench,一个组合视频编辑基准,包含416个精心策划的源视频、626条多点编辑指令和9,990个细粒度检查项。CoVEBench覆盖多样化的编辑维度,通过MLLM评判的指令遵循度和视频保真度,以及视频质量的自动指标来评估模型。大量实验表明,组合编辑仍然是一个深层次的挑战:当前模型在处理多个操作同时进行时,经常遗漏编辑、违反保留约束或引入伪影。CoVEBench为推进视频编辑向现实用户工作流发展提供了一个具有挑战性的诊断测试平台。
英文摘要:
While recent text-guided video editing models excel at elementary tasks (e.g., style transfer, object insertion), real-world user requests are highly compositional. A single prompt often demands multiple coupled edits, such as modifying subjects, actions, and camera views, while strictly preserving unrelated spatiotemporal content. Existing benchmarks, heavily constrained by isolated edits and coarse global metrics, fail to diagnose how models handle such complex workflows. To address this gap, we introduce CoVEBench, a compositional video editing benchmark comprising 416 curated source videos, 626 multi-point editing instructions, and 9,990 fine-grained checklist items. Covering diverse editing dimensions, CoVEBench evaluates models via MLLM-judged instruction compliance and video fidelity, alongside automated metrics for video quality. Extensive experiments reveal that compositional editing remains a profound challenge: current models frequently omit edits, violate preservation constraints, or introduce artifacts when handling multiple operations simultaneously. CoVEBench provides a challenging, diagnostic testbed to advance video editing toward realistic user workflows.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。