面向统一模型的基于推理的视频编辑：带有自我反思学习
Towards Reason-Informed Video Editing in Unified Models with Self-Reflective Learning
AI总结：
本文提出RVE任务，通过构建RVE-Bench基准，引入自反思学习框架ReViSE，提升视频编辑的准确性和视觉质量。
摘要：
统一视频模型在理解和生成方面表现出色，但在配备强大内部视觉语言模型（VLMs）的情况下仍难以实现基于推理的视觉编辑。我们归因于两个因素：（1）现有数据集不足以训练和评估具有推理能力的视频编辑；（2）模型推理与编辑能力之间存在固有断层，阻碍了理解指导编辑过程。为此，我们引入了基于推理的视频编辑（RVE）任务，要求在编辑过程中推理物理可能性和因果动态。为支持系统评估，我们构建了RVE-Bench基准，包含两个互补子集：推理感知视频编辑（RAVE）和上下文视频到视频生成（ICVG），覆盖编辑和生成场景中的多样化推理维度。基于此基础，我们提出了ReViSE，一种自反思学习框架，利用模型内部VLM评估和优化其生成。不同于以往依赖外部批评者的奖励方法，ReViSE利用模型内部VLM作为自我反思评估者，提供可微分反馈，直接在训练过程中优化生成器的推理行为。在RVE-Bench上的广泛实验表明，ReViSE提升了编辑准确性和视觉保真度，其在RAVE子集的总体得分比微调对照组高出10%，证明了自反思可微分奖励的有效性。
英文摘要：
Unified video models exhibit strong capabilities in understanding and generation, yet they struggle with reason-informed visual editing even when equipped with powerful internal vision-language models (VLMs). We attribute this gap to two factors: (1) existing datasets are inadequate for training and evaluating reasoning-aware video editing, and (2) an inherent disconnect between the models' reasoning and editing capabilities, which prevents understanding from guiding the editing process. To address this, we introduce the Reason-Informed Video Editing (RVE) task, which requires reasoning about physical plausibility and causal dynamics during editing. To support systematic evaluation, we construct RVE-Bench, a comprehensive benchmark with two complementary subsets: Reasoning-Aware Video Editing (RAVE) and In-Context Video-to-Video Generation (ICVG), spanning diverse reasoning dimensions across both editing and generation scenarios. Building upon this foundation, we propose ReViSE, a self-reflective learning framework that harnesses the model's internal VLM to evaluate and refine its own generation during training. Unlike prior reward-based approaches that rely on external critics, ReViSE leverages the model's internal VLM as a self-reflective evaluator, providing differentiable feedback that directly refines the generator's reasoning behavior during training. Extensive experiments on RVE-Bench demonstrate that ReViSE enhances editing accuracy and visual fidelity, outperforming the finetuned counterpart by 10% in Overall score on the RAVE subset, demonstrating the effectiveness of self-reflective differentiable reward.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
