SmoothVideo：基于扩散模型噪声约束的平滑视频合成，用于一次性视频调优
SmoothVideo: Smooth Video Synthesis with Noise Constraints on Diffusion Models for One-shot Video Tuning
AI总结：
针对一次性视频调优中视频不连贯问题，提出跨帧噪声约束损失以平滑潜在表示，并引入新评估指标，显著提升生成视频的一致性和平滑度。
摘要：
近期的一次性视频调优方法，即基于预训练的文生图模型（如Stable Diffusion）在特定视频上对网络进行微调，因其灵活性而广受欢迎。然而，这些方法生成的视频常出现不连贯和不一致的问题。为解决这些局限，本文提出了一种简单而有效的跨视频帧噪声约束。该约束旨在调节噪声预测在其时间邻域内的一致性，从而产生平滑的潜在表示。它可简单地作为训练阶段的一个损失项加入。通过将该损失应用于现有的一次性视频调优方法，我们显著提升了生成视频的整体一致性和平滑度。此外，我们认为当前的视频评估指标未能充分捕捉平滑度。为此，我们引入了一种考虑细节特征及其时间动态的新指标。实验结果验证了我们的方法在各种一次性视频调优基线上生成更平滑视频的有效性。源代码和视频演示可在https://github.com/SPengLiang/SmoothVideo获取。
英文摘要：
Recent one-shot video tuning methods, which fine-tune the network on a specific video based on pre-trained text-to-image models (e.g., Stable Diffusion), are popular in the community because of the flexibility. However, these methods often produce videos marred by incoherence and inconsistency. To address these limitations, this paper introduces a simple yet effective noise constraint across video frames. This constraint aims to regulate noise predictions across their temporal neighbors, resulting in smooth latents. It can be simply included as a loss term during the training phase. By applying the loss to existing one-shot video tuning methods, we significantly improve the overall consistency and smoothness of the generated videos. Furthermore, we argue that current video evaluation metrics inadequately capture smoothness. To address this, we introduce a novel metric that considers detailed features and their temporal dynamics. Experimental results validate the effectiveness of our approach in producing smoother videos on various one-shot video tuning baselines. The source codes and video demos are available at \href{https://github.com/SPengLiang/SmoothVideo}{https://github.com/SPengLiang/SmoothVideo}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
