时间反演采样中用于生成中间帧的运动先验蒸馏
Motion Prior Distillation in Time Reversal Sampling for Generative Inbetweening
AI总结：
本文提出运动先验蒸馏技术，通过蒸馏正向路径的运动残差到反向路径，以减少时间反演采样中的不连续性和伪影，提升生成中间帧的连贯性。
摘要：
最近图像到视频（I2V）扩散模型的进步显著推动了生成中间帧领域的进展，该领域旨在生成两个关键帧之间的语义合理的帧。特别是，在推理时间采样策略中，利用大规模预训练I2V模型的生成先验，而无需额外训练，已成为一种越来越受欢迎的方法。然而，现有的推理时间采样，无论是并行融合正向和反向路径还是顺序交替，往往由于两个生成路径之间的不对齐而产生时间不连续性和不希望的视觉伪影。这是因为每条路径都遵循由其自身条件帧诱导的运动先验。在本文中，我们提出了运动先验蒸馏（MPD），一种简单而有效的推理时间蒸馏技术，通过将正向路径的运动残差蒸馏到反向路径中，来抑制双向不匹配。我们的方法可以有意避免对终点条件路径进行去噪，从而避免路径的歧义，并通过正向运动先验产生更具时间连贯性的中间帧结果。我们不仅在标准基准上进行了定量评估，还进行了广泛的用户研究，以证明我们的方法在实际场景中的有效性。
英文摘要：
Recent progress in image-to-video (I2V) diffusion models has significantly advanced the field of generative inbetweening, which aims to generate semantically plausible frames between two keyframes. In particular, inference-time sampling strategies, which leverage the generative priors of large-scale pre-trained I2V models without additional training, have become increasingly popular. However, existing inference-time sampling, either fusing forward and backward paths in parallel or alternating them sequentially, often suffers from temporal discontinuities and undesirable visual artifacts due to the misalignment between the two generated paths. This is because each path follows the motion prior induced by its own conditioning frame. In this work, we propose Motion Prior Distillation (MPD), a simple yet effective inference-time distillation technique that suppresses bidirectional mismatch by distilling the motion residual of the forward path into the backward path. Our method can deliberately avoid denoising the end-conditioned path which causes the ambiguity of the path, and yield more temporally coherent inbetweening results with the forward motion prior. We not only perform quantitative evaluations on standard benchmarks, but also conduct extensive user studies to demonstrate the effectiveness of our approach in practical scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
