PipeFlow: 管道处理与运动感知帧选择用于长格式视频编辑
PipeFlow: Pipelined Processing and Motion-Aware Frame Selection for Long-Form Video Editing
AI总结：
PipeFlow通过运动感知帧选择和流水线任务调度，实现长格式视频编辑的高效处理，相比现有方法速度提升达9.6至31.7倍。
摘要：
长格式视频编辑由于联合编辑和去噪扩散隐式模型（DDIM）反转在扩展序列中的计算成本呈指数增长而面临独特挑战。为了解决这些限制，我们提出了PipeFlow，一种可扩展的流水线视频编辑方法，引入了三个关键创新：首先，基于结构相似性指数度量（SSIM）和光流的运动分析，我们识别并提出跳过低运动帧的编辑。其次，我们提出了一种流水线任务调度算法，将视频分成多个片段，并根据可用GPU内存并行执行DDIM反转和联合编辑。最后，我们利用基于神经网络的插值技术来平滑段之间边框帧，并插值之前跳过的帧。我们的方法通过将视频分成更小的片段，能够扩展到更长的视频，使PipeFlow的编辑时间与视频长度成线性关系。原则上，这使能够编辑无限长的视频而无需像其他方法那样遇到递增的每帧计算开销。PipeFlow相比TokenFlow实现了高达9.6倍的速度提升，相比扩散运动转移（DMT）实现了31.7倍的速度提升。
英文摘要：
Long-form video editing poses unique challenges due to the exponential increase in the computational cost from joint editing and Denoising Diffusion Implicit Models (DDIM) inversion across extended sequences. To address these limitations, we propose PipeFlow, a scalable, pipelined video editing method that introduces three key innovations: First, based on a motion analysis using Structural Similarity Index Measure (SSIM) and Optical Flow, we identify and propose to skip editing of frames with low motion. Second, we propose a pipelined task scheduling algorithm that splits a video into multiple segments and performs DDIM inversion and joint editing in parallel based on available GPU memory. Lastly, we leverage a neural network-based interpolation technique to smooth out the border frames between segments and interpolate the previously skipped frames. Our method uniquely scales to longer videos by dividing them into smaller segments, allowing PipeFlow's editing time to increase linearly with video length. In principle, this enables editing of infinitely long videos without the growing per-frame computational overhead encountered by other methods. PipeFlow achieves up to a 9.6X speedup compared to TokenFlow and a 31.7X speedup over Diffusion Motion Transfer (DMT).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
