arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.13413cs.CV
收藏

TVG:一种基于扩散模型的无训练过渡视频生成方法

TVG: A Training-free Transition Video Generation Method with Diffusion Models

Sobey Media Intelligence Laboratory(索贝媒体智能实验室)

Rui Zhang, Yaosen Chen, Yuegen Liu, Wei Wang, Xuming Wen, Hongxia Wang

AI总结:

提出无训练的TVG方法,利用高斯过程回归、插值条件控制与FBiF架构,基于视频级扩散模型生成高质量平滑过渡视频。

摘要:

过渡视频在媒体制作中起着至关重要的作用,能增强视觉叙事的流畅性和连贯性。传统方法如变形往往缺乏艺术吸引力且需要专业技能,限制了其效果。近期基于扩散模型的视频生成进展为创建过渡提供了新可能,但面临帧间关系建模差和内容突变等挑战。我们提出了一种新颖的无训练过渡视频生成(TVG)方法,使用视频级扩散模型,无需额外训练即可解决这些限制。该方法利用高斯过程回归($\mathcal{GPR}$)对潜在表示进行建模,确保帧间过渡平滑且具动态感。此外,我们引入基于插值的条件控制和频率感知双向融合(FBiF)架构,以增强时间控制和过渡可靠性。对基准数据集和自定义图像对的评估证明了该方法在生成高质量平滑过渡视频方面的有效性。代码见 https://sobeymil.github.io/tvg.com。

英文摘要:

Transition videos play a crucial role in media production, enhancing the flow and coherence of visual narratives. Traditional methods like morphing often lack artistic appeal and require specialized skills, limiting their effectiveness. Recent advances in diffusion model-based video generation offer new possibilities for creating transitions but face challenges such as poor inter-frame relationship modeling and abrupt content changes. We propose a novel training-free Transition Video Generation (TVG) approach using video-level diffusion models that addresses these limitations without additional training. Our method leverages Gaussian Process Regression ($\mathcal{GPR}$) to model latent representations, ensuring smooth and dynamic transitions between frames. Additionally, we introduce interpolation-based conditional controls and a Frequency-aware Bidirectional Fusion (FBiF) architecture to enhance temporal control and transition reliability. Evaluations of benchmark datasets and custom image pairs demonstrate the effectiveness of our approach in generating high-quality smooth transition videos. The code are provided in https://sobeymil.github.io/tvg.com.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑