Follow-Your-Canvas：具有广泛内容生成的高分辨率视频外扩
Follow-Your-Canvas: Higher-Resolution Video Outpainting with Extensive Content Generation
AI总结：
本文提出Follow-Your-Canvas方法，通过空间窗口分布与无缝合并、注入源视频位置关系，突破GPU内存限制，实现高分辨率、大规模且内容丰富的视频外扩。
摘要：
本文探索了具有广泛内容生成的高分辨率视频外扩。我们指出现有方法在尝试大幅外扩视频时面临的常见问题：低质量内容的生成以及GPU内存带来的限制。为应对这些挑战，我们提出了一种基于扩散的方法，称为Follow-Your-Canvas。它建立在两个核心设计之上。首先，我们不采用常见的“单次”外扩做法，而是将任务分布到空间窗口中并无缝合并它们。这使我们能够外扩任意尺寸和分辨率的视频，而不受GPU内存的约束。其次，源视频及其相对位置关系被注入到每个窗口的生成过程中。这使得每个窗口内生成的空间布局与源视频相协调。结合这两个设计，我们能够在保持空间和时间一致性的同时，生成具有丰富内容的高分辨率外扩视频。Follow-Your-Canvas在大规模视频外扩中表现出色，例如从512X512到1152X2048（9倍），同时产生高质量且美观的结果。它在各种分辨率和尺度设置下取得了最佳的定量结果。代码已发布在https://github.com/mayuelala/FollowYourCanvas。
英文摘要：
This paper explores higher-resolution video outpainting with extensive content generation. We point out common issues faced by existing methods when attempting to largely outpaint videos: the generation of low-quality content and limitations imposed by GPU memory. To address these challenges, we propose a diffusion-based method called \textit{Follow-Your-Canvas}. It builds upon two core designs. First, instead of employing the common practice of "single-shot" outpainting, we distribute the task across spatial windows and seamlessly merge them. It allows us to outpaint videos of any size and resolution without being constrained by GPU memory. Second, the source video and its relative positional relation are injected into the generation process of each window. It makes the generated spatial layout within each window harmonize with the source video. Coupling with these two designs enables us to generate higher-resolution outpainting videos with rich content while keeping spatial and temporal consistency. Follow-Your-Canvas excels in large-scale video outpainting, e.g., from 512X512 to 1152X2048 (9X), while producing high-quality and aesthetically pleasing results. It achieves the best quantitative results across various resolution and scale setups. The code is released on https://github.com/mayuelala/FollowYourCanvas
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
