Ada-VE：使用自适应运动先验的免训练一致视频编辑
Ada-VE: Training-Free Consistent Video Editing Using Adaptive Motion Prior
AI总结：
本文提出Ada-VE，一种免训练的自适应运动引导跨帧注意力机制，利用光流减少冗余计算，在相同预算下处理三倍帧数，并引入KV缓存提升视频编辑的一致性与质量。
摘要：
视频到视频合成在保持角色一致性、平滑的时间过渡以及在快速运动过程中保持视觉质量方面面临重大挑战。虽然最近的完全跨帧自注意力机制提高了多帧之间的角色一致性，但它们带来了高计算成本，并且通常包含冗余操作，特别是对于具有较高帧率的视频。为了解决这些低效问题，我们提出了一种自适应运动引导的跨帧注意力机制，该机制选择性地减少冗余计算。这使得在相同的计算预算内，能够在更多帧上执行更多的跨帧注意力，从而增强视频质量和时间连贯性。我们的方法利用光流来聚焦于运动区域，同时对静止区域进行稀疏关注，从而在不增加计算需求的情况下实现更多帧的联合编辑。传统的帧插值技术在中间帧中难以处理运动模糊和闪烁，这损害了视觉保真度。为了缓解这一问题，我们引入了用于联合编辑帧的KV缓存，在中间帧之间重用键和值，以保持视觉质量并在整个视频中维持时间一致性。通过我们的自适应跨帧自注意力方法，与现有方法相比，我们在相同的计算预算下，在完全跨帧注意力基线的计算预算内，将处理的帧数增加了三倍。这导致预测准确性和时间一致性显著提高，优于最先进的方法。代码将在https://github.com/tanvir-utexas/AdaVE/tree/main公开提供。
英文摘要：
Video-to-video synthesis poses significant challenges in maintaining character consistency, smooth temporal transitions, and preserving visual quality during fast motion. While recent fully cross-frame self-attention mechanisms have improved character consistency across multiple frames, they come with high computational costs and often include redundant operations, especially for videos with higher frame rates. To address these inefficiencies, we propose an adaptive motion-guided cross-frame attention mechanism that selectively reduces redundant computations. This enables a greater number of cross-frame attentions over more frames within the same computational budget, thereby enhancing both video quality and temporal coherence. Our method leverages optical flow to focus on moving regions while sparsely attending to stationary areas, allowing for the joint editing of more frames without increasing computational demands. Traditional frame interpolation techniques struggle with motion blur and flickering in intermediate frames, which compromises visual fidelity. To mitigate this, we introduce KV-caching for jointly edited frames, reusing keys and values across intermediate frames to preserve visual quality and maintain temporal consistency throughout the video. With our adaptive cross-frame self-attention approach, we achieve a threefold increase in the number of keyframes processed compared to existing methods, all within the same computational budget as fully cross-frame attention baselines. This results in significant improvements in prediction accuracy and temporal consistency, outperforming state-of-the-art approaches. Code will be made publicly available at https://github.com/tanvir-utexas/AdaVE/tree/main
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
