MiVE：用于参考引导视频编辑的多尺度视觉语言特征
MiVE: Multiscale Vision-language features for reference-guided video Editing
AI总结：
提出MiVE框架，利用VLM的多尺度层次特征（早期层保留空间细节，深层编码全局语义）统一到自注意力扩散Transformer中，解决模态间隙和细粒度信息丢失问题，在参考引导视频编辑中达到SOTA性能。
摘要：
参考引导视频编辑以源视频、文本指令和参考图像作为输入，要求模型在忠实执行指令编辑的同时保留原始运动及未编辑内容。现有方法分为两种范式，各有固有限制：解耦编码器在处理指令和视觉内容时存在模态间隙，而统一视觉语言编码器仅依赖最终层表示，丢失了细粒度空间细节。我们观察到VLM层层次化地编码互补信息——早期层捕获局部空间细节，对精确编辑至关重要；深层编码全局语义，用于指令理解。基于此洞察，我们提出MiVE（用于参考引导视频编辑的多尺度视觉语言特征），该框架将VLM重新用作多尺度特征提取器。MiVE从Qwen3-VL提取层次特征，并将其集成到统一的自注意力扩散Transformer中，消除了交叉注意力设计中固有的模态不匹配。实验表明，MiVE在人类偏好中排名最高，性能优于学术方法和商业系统，达到了最先进水平。
英文摘要：
Reference-guided video editing takes a source video, a text instruction, and a reference image as inputs, requiring the model to faithfully apply the instructed edits while preserving original motion and unedited content. Existing methods fall into two paradigms, each with inherent limitations: decoupled encoders suffer from modality gaps when processing instructions and visual content independently, while unified vision-language encoders lose fine-grained spatial details by relying solely on final-layer representations. We observe that VLM layers encode complementary information hierarchically -- early layers capture localized spatial details essential for precise editing, while deeper layers encode global semantics for instruction comprehension. Building on this insight, we present MiVE (Multiscale Vision-language features for reference-guided video Editing), a framework that repurposes VLMs as multiscale feature extractors. MiVE extracts hierarchical features from Qwen3-VL and integrates them into a unified self-attention Diffusion Transformer, eliminating the modality mismatch inherent in cross-attention designs. Experiments demonstrate that MiVE achieves state-of-the-art performance by ranking highest in human preference, outperforming both academic methods and commercial systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
