arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.19435cs.CV

图像到视频模型是否是良好的零样本图像编辑器?

Are Image-to-Video Models Good Zero-Shot Image Editors?

Zhejiang University(浙江大学);Harvard University(哈佛大学)

Zechuan Zhang, Zhenyuan Chen, Zongxin Yang, Yi Yang

AI总结:

IF-Edit通过无需微调的框架,利用预训练图像到视频扩散模型实现指令驱动的图像编辑,解决提示错位、冗余时间潜在特征和模糊后期帧问题,展现强大的推理能力和通用编辑竞争力。

摘要:

大规模视频扩散模型表现出强大的世界模拟和时间推理能力,但将其用作零样本图像编辑器仍处于探索阶段。我们介绍了IF-Edit,一种无需微调的框架,该框架重新利用预训练的图像到视频扩散模型进行指令驱动的图像编辑。IF-Edit解决三个关键挑战:提示错位、冗余时间潜在特征和模糊的后期帧。它包括(1)一个链式思维提示增强模块,将静态编辑指令转换为时间相关的推理提示;(2)一个时间潜在特征dropout策略,在专家切换点后压缩帧潜在特征,加速去噪同时保持语义和时间一致性;(3)一个自洽的后细化步骤,利用短的静态视频轨迹来锐化后期帧。在四个公开基准测试上进行的实验,涵盖非刚性编辑、物理和时间推理以及通用指令编辑,表明IF-Edit在以推理为中心的任务上表现强劲,同时在通用任务上保持竞争力。我们的研究为视频扩散模型作为图像编辑器提供了系统视角,并突显了统一视频-图像生成推理的简单配方。

英文摘要:

Large-scale video diffusion models show strong world simulation and temporal reasoning abilities, but their use as zero-shot image editors remains underexplored. We introduce IF-Edit, a tuning-free framework that repurposes pretrained image-to-video diffusion models for instruction-driven image editing. IF-Edit addresses three key challenges: prompt misalignment, redundant temporal latents, and blurry late-stage frames. It includes (1) a chain-of-thought prompt enhancement module that transforms static editing instructions into temporally grounded reasoning prompts; (2) a temporal latent dropout strategy that compresses frame latents after the expert-switch point, accelerating denoising while preserving semantic and temporal coherence; and (3) a self-consistent post-refinement step that sharpens late-stage frames using a short still-video trajectory. Experiments on four public benchmarks, covering non-rigid editing, physical and temporal reasoning, and general instruction edits, show that IF-Edit performs strongly on reasoning-centric tasks while remaining competitive on general-purpose edits. Our study provides a systematic view of video diffusion models as image editors and highlights a simple recipe for unified video-image generative reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑