融合你的潜变量：基于多源潜扩散模型的视频编辑
Fuse Your Latents: Video Editing with Multi-source Latent Diffusion Models
AI总结：
针对文本到视频编辑存在的时间一致性差、结构不佳等问题，提出无需训练的FLDM框架，通过带调度超参数融合图像与视频潜变量，实现两类LDM互补，可作为通用插件提升现成模型的视频编辑质量，效果优于现有最优方法。
摘要：
潜扩散模型（Latent Diffusion Models, LDMs）以其在图像和视频合成领域的强大能力著称。然而，与文本到图像（text-to-image, T2I）编辑相比，文本到视频（text-to-video, T2V）编辑受预训练数据不足、模型可编辑性有限或调优成本过高的影响，存在时间一致性和结构表现不佳的问题。为填补这一空白，我们提出FLDM（Fused Latent Diffusion Model，融合潜扩散模型）——一种无需训练的框架，通过整合多种T2I和T2V LDM实现高质量的T2V编辑。具体而言，FLDM利用带更新调度的超参数，在去噪过程中有效融合图像与视频潜变量。本文首次揭示T2I与T2V LDM可在结构和时间一致性上形成互补，最终生成高质量视频。值得注意的是，FLDM可作为通用插件，适用于现成的图像与视频LDM，显著提升视频编辑质量。在主流T2I和T2V LDM上开展的大量定量与定性实验表明，FLDM的编辑质量优于当前最优的T2V编辑方法。我们的项目代码可在https://github.com/lutianyi0603/fuse_your_latents获取。
英文摘要：
Latent Diffusion Models (LDMs) are renowned for their powerful capabilities in image and video synthesis. Yet, compared to text-to-image (T2I) editing, text-to-video (T2V) editing suffers from a lack of decent temporal consistency and structure, due to insufficient pre-training data, limited model editability, or extensive tuning costs. To address this gap, we propose FLDM (Fused Latent Diffusion Model), a training-free framework that achieves high-quality T2V editing by integrating various T2I and T2V LDMs. Specifically, FLDM utilizes a hyper-parameter with an update schedule to effectively fuse image and video latents during the denoising process. This paper is the first to reveal that T2I and T2V LDMs can complement each other in terms of structure and temporal consistency, ultimately generating high-quality videos. It is worth noting that FLDM can serve as a versatile plugin, applicable to off-the-shelf image and video LDMs, to significantly enhance the quality of video editing. Extensive quantitative and qualitative experiments on popular T2I and T2V LDMs demonstrate FLDM's superior editing quality than state-of-the-art T2V editing methods. Our project code is available at https://github.com/lutianyi0603/fuse_your_latents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
