SeedVR：在扩散Transformer中植入无限生成能力以实现通用视频修复
SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration
AI总结：
针对现有扩散视频修复方法生成能力与采样效率有限的问题，提出带移位窗口注意力的扩散Transformer SeedVR，支持任意长度分辨率视频修复，在多类基准上性能优于现有方法。
摘要：
视频修复在从真实场景中的未知退化里恢复时间一致细节的同时保持保真度，面临着不小的挑战。尽管近期基于扩散的修复技术取得了进展，但这类方法往往在生成能力和采样效率上存在局限。本研究提出SeedVR，一款旨在处理任意长度和分辨率的真实世界视频修复任务的扩散Transformer。SeedVR的核心设计是移位窗口注意力，可助力长视频序列实现有效修复。SeedVR还支持在空间和时间维度边界附近使用可变尺寸窗口，突破了传统窗口注意力的分辨率限制。结合因果视频自编码器、图文混合训练、渐进式训练等当前主流技术，SeedVR在合成基准、真实世界基准以及AI生成视频上均取得了极具竞争力的性能。大量实验证明，SeedVR在通用视频修复任务上优于现有方法。
英文摘要：
Video restoration poses non-trivial challenges in maintaining fidelity while recovering temporally consistent details from unknown degradations in the wild. Despite recent advances in diffusion-based restoration, these methods often face limitations in generation capability and sampling efficiency. In this work, we present SeedVR, a diffusion transformer designed to handle real-world video restoration with arbitrary length and resolution. The core design of SeedVR lies in the shifted window attention that facilitates effective restoration on long video sequences. SeedVR further supports variable-sized windows near the boundary of both spatial and temporal dimensions, overcoming the resolution constraints of traditional window attention. Equipped with contemporary practices, including causal video autoencoder, mixed image and video training, and progressive training, SeedVR achieves highly-competitive performance on both synthetic and real-world benchmarks, as well as AI-generated videos. Extensive experiments demonstrate SeedVR's superiority over existing methods for generic video restoration.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
