流式视频扩散：使用扩散模型的在线视频编辑
Streaming Video Diffusion: Online Video Editing with Diffusion Models
AI总结：
提出在线视频编辑任务以实现流式帧的时间一致性编辑，并构建SVDiff模型，通过整合空间感知时间递归到Stable Diffusion并采用分段级训练，在512x512分辨率下实现15.2 FPS的实时高质量长视频编辑。
摘要：
我们提出了一项名为在线视频编辑的新颖任务，旨在编辑流式帧的同时保持时间一致性。与假设所有帧均已预先建立且可访问的现有离线视频编辑不同，在线视频编辑专为直播和在线聊天等现实应用而定制，需要满足：(1) 快速的连续步推理，(2) 长期时间建模，以及 (3) 零样本视频编辑能力。为解决这些问题，我们提出了流式视频扩散（SVDiff），它将紧凑的空间感知时间递归整合到现成的 Stable Diffusion 中，并在大规模长视频上以分段级方案进行训练。这种简单而有效的设置使我们能够获得一个单一模型，该模型能够处理广泛的视频，并以时间连贯性编辑每一流式帧。我们的实验表明，我们的模型能够以显著的效果编辑长且高质量的视频，在 512x512 的分辨率下实现了 15.2 FPS 的实时推理速度。
英文摘要：
We present a novel task called online video editing, which is designed to edit \textbf{streaming} frames while maintaining temporal consistency. Unlike existing offline video editing assuming all frames are pre-established and accessible, online video editing is tailored to real-life applications such as live streaming and online chat, requiring (1) fast continual step inference, (2) long-term temporal modeling, and (3) zero-shot video editing capability. To solve these issues, we propose Streaming Video Diffusion (SVDiff), which incorporates the compact spatial-aware temporal recurrence into off-the-shelf Stable Diffusion and is trained with the segment-level scheme on large-scale long videos. This simple yet effective setup allows us to obtain a single model that is capable of executing a broad range of videos and editing each streaming frame with temporal coherence. Our experiments indicate that our model can edit long, high-quality videos with remarkable results, achieving a real-time inference speed of 15.2 FPS at a resolution of 512x512.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
