VRWKV-Editor: 降低基于变换器的视频编辑中的二次复杂度
VRWKV-Editor: Reducing quadratic complexity in transformer-based video editing
AI总结：
VRWKV-Editor通过引入线性时空聚合模块，降低基于变换器的视频编辑模型的计算复杂度，实现3.7倍加速和60%内存节省，同时保持高质量的帧一致性和文本对齐。
摘要：
在近期视频编辑领域取得进展的背景下，关注空间和时间依赖性的深度学习模型已成为主要方法。然而，这些模型受到传统注意力机制二次计算复杂度的限制，难以适应长时长和高分辨率视频。这一限制限制了它们在实时视频处理等实际场景中的应用。为了解决这一挑战，我们提出了一种方法，通过引入VRWKV-Editor，一种新颖的视频编辑模型，将线性时空聚合模块整合到基于视频的扩散模型中。VRWKV-Editor利用RWKV变换器的双向加权键值递归机制，捕捉全局依赖性的同时保持时间一致性，实现线性复杂度而不牺牲质量。大量实验表明，所提出的方法在与最先进的扩散视频编辑方法相比，实现了高达3.7倍的加速和60%的内存使用减少，同时在帧一致性和文本对齐方面保持了竞争性的性能。此外，我们在不同序列长度的视频上进行的比较分析证实，随着视频长度增加，我们的方法与具有自注意力机制的架构之间的编辑速度差距变得更加显著。
英文摘要：
In light of recent progress in video editing, deep learning models focusing on both spatial and temporal dependencies have emerged as the primary method. However, these models suffer from the quadratic computational complexity of traditional attention mechanisms, making them difficult to adapt to long-duration and high-resolution videos. This limitation restricts their applicability in practical contexts such as real-time video processing. To tackle this challenge, we introduce a method to reduce both time and space complexity of these systems by proposing VRWKV-Editor, a novel video editing model that integrates a linear spatio-temporal aggregation module into video-based diffusion models. VRWKV-Editor leverages bidirectional weighted key-value recurrence mechanism of the RWKV transformer to capture global dependencies while preserving temporal coherence, achieving linear complexity without sacrificing quality. Extensive experiments demonstrate that the proposed method achieves up to 3.7x speedup and 60% lower memory usage compared to state-of-the-art diffusion-based video editing methods, while maintaining competitive performance in frame consistency and text alignment. Furthermore, a comparative analysis we conducted on videos with different sequence lengths confirms that the gap in editing speed between our approach and architectures with self-attention becomes more significant with long videos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
