FreeViS:面向不一致参考的免训练视频风格化
FreeViS: Training-free Video Stylization with Inconsistent References
AI总结:
针对视频风格化存在的时间一致性差、训练成本高的问题,提出免训练框架FreeViS,通过整合多参考图到预训练I2V模型、结合高频补偿与光流运动线索,实现了高保真、强时间连贯的风格化效果,性能优于现有基线。
摘要:
视频风格化在内容创作中发挥着关键作用,但仍是一个具有挑战性的问题。逐帧直接应用图像风格化会损害时间一致性,降低风格丰富度。另一种方案是训练专用的视频风格化模型,这通常需要成对的视频数据,且计算成本高昂。本文提出FreeViS,一种免训练的视频风格化框架,可生成风格细节丰富、时间连贯性强的风格化视频。我们的方法将多个风格化参考图整合到预训练的图像到视频(I2V)模型中,有效缓解了以往工作中存在的传播误差,且不会引入闪烁和卡顿。此外,该方法利用高频补偿约束内容布局与运动,结合基于光流的运动线索,在低显著性区域保留风格纹理。经过大量评估,FreeViS实现了更高的风格化保真度与更优的时间一致性,性能优于近期基线方法,并获得了强烈的人类偏好。我们的免训练流程为高质量、时间连贯的视频风格化提供了实用且经济的解决方案。代码与视频可通过https://xujiacong.github.io/FreeViS/获取。
英文摘要:
Video stylization plays a key role in content creation, but it remains a challenging problem. Naïvely applying image stylization frame-by-frame hurts temporal consistency and reduces style richness. Alternatively, training a dedicated video stylization model typically requires paired video data and is computationally expensive. In this paper, we propose FreeViS, a training-free video stylization framework that generates stylized videos with rich style details and strong temporal coherence. Our method integrates multiple stylized references to a pretrained image-to-video (I2V) model, effectively mitigating the propagation errors observed in prior works, without introducing flickers and stutters. In addition, it leverages high-frequency compensation to constrain the content layout and motion, together with flow-based motion cues to preserve style textures in low-saliency regions. Through extensive evaluations, FreeViS delivers higher stylization fidelity and superior temporal consistency, outperforming recent baselines and achieving strong human preference. Our training-free pipeline offers a practical and economic solution for high-quality, temporally coherent video stylization. The code and videos can be accessed via https://xujiacong.github.io/FreeViS/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。