arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.01720cs.CV

FFP-300K:用于通用视频编辑的可扩展首帧传播

FFP-300K: Scaling First-Frame Propagation for Generalizable Video Editing

FDU(阜新大学);Tencent YouTu Lab(腾讯优图实验室);Shanghai Innovation Institute(上海创新研究院)

Xijie Huang, Chengming Xu, Donghao Luo, Xiaobin Hu, Peng Tang, Xu Peng, Jiangning Zhang, Chengjie Wang, Yanwei Fu

AI总结:

FFP-300K通过构建大规模数据集和改进框架,实现了无指导的首帧传播,提升视频编辑的稳定性和性能。

摘要:

首帧传播(FFP)提供了一种有前景的可控视频编辑范式,但现有方法受到对复杂运行时指导的依赖所限制。我们识别出这一限制的根本原因在于当前训练数据集的不足,这些数据集通常太短、低分辨率,并且缺乏教导鲁棒时间先验所需的任务多样性。为了解决这一基础数据缺口,我们首先引入了FFP-300K,这是一个包含30万对高保真视频的大型数据集,分辨率为720p,长度为81帧,通过一个原则性的双轨流程构建,用于多样化的局部和全局编辑。基于此数据集,我们提出了一种新的框架,用于真正的无指导FFP,解决了在保持首帧外观和保留源视频运动之间的关键张力。在架构上,我们引入了自适应时空RoPE(AST-RoPE),它动态地重新映射位置编码,以解耦外观和运动参考。在目标层面,我们采用了一种自我蒸馏策略,其中身份传播任务作为强大的正则化器,确保长期时间稳定性和防止语义漂移。在EditVerseBench基准上的全面实验表明,我们的方法显著优于现有学术和商业模型,通过获得约0.2 PickScore和0.3 VLM分数的改进,这些竞争对手。

英文摘要:

First-Frame Propagation (FFP) offers a promising paradigm for controllable video editing, but existing methods are hampered by a reliance on cumbersome run-time guidance. We identify the root cause of this limitation as the inadequacy of current training datasets, which are often too short, low-resolution, and lack the task diversity required to teach robust temporal priors. To address this foundational data gap, we first introduce FFP-300K, a new large-scale dataset comprising 300K high-fidelity video pairs at 720p resolution and 81 frames in length, constructed via a principled two-track pipeline for diverse local and global edits. Building on this dataset, we propose a novel framework designed for true guidance-free FFP that resolves the critical tension between maintaining first-frame appearance and preserving source video motion. Architecturally, we introduce Adaptive Spatio-Temporal RoPE (AST-RoPE), which dynamically remaps positional encodings to disentangle appearance and motion references. At the objective level, we employ a self-distillation strategy where an identity propagation task acts as a powerful regularizer, ensuring long-term temporal stability and preventing semantic drift. Comprehensive experiments on the EditVerseBench benchmark demonstrate that our method significantly outperforming existing academic and commercial models by receiving about 0.2 PickScore and 0.3 VLM score improvement against these competitors.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑