FilmWeaver: 通过缓存引导自回归扩散编织一致的多镜头视频
FilmWeaver: Weaving Consistent Multi-Shot Videos with Cache-Guided Autoregressive Diffusion
AI总结：
FilmWeaver通过缓存引导自回归扩散模型实现多镜头视频的一致性生成，支持灵活交互和多任务应用。
摘要：
当前视频生成模型在单镜头合成方面表现良好，但在多镜头视频生成方面面临重大挑战，特别是在保持镜头间角色和背景一致性以及灵活生成任意长度和镜头数量的视频方面存在困难。为了解决这些限制，我们引入了FilmWeaver，一种新的框架，旨在生成任意长度的一致多镜头视频。首先，它采用自回归扩散范式实现任意长度的视频生成。为了解决一致性问题，我们的关键见解是将问题分解为镜头间一致性和镜头内连贯性。我们通过双层缓存机制实现这一点：镜头记忆缓存先前镜头的关键帧以保持角色和场景身份，而时间记忆保留当前镜头的帧历史以确保流畅连续的运动。所提出的框架允许灵活的、多轮用户交互来创建多镜头视频。此外，由于这种解耦设计，我们的方法通过支持多概念注入和视频扩展等下游任务展示了高度的通用性。为了促进我们一致性感知方法的训练，我们还开发了一个全面的管道来构建高质量的多镜头视频数据集。广泛的实验结果表明，我们的方法在一致性和审美质量指标上都超过了现有方法，为创建更加一致、可控和叙事驱动的视频内容开辟了新可能。项目页面：https://filmweaver.github.io
英文摘要：
Current video generation models perform well at single-shot synthesis but struggle with multi-shot videos, facing critical challenges in maintaining character and background consistency across shots and flexibly generating videos of arbitrary length and shot count. To address these limitations, we introduce \textbf{FilmWeaver}, a novel framework designed to generate consistent, multi-shot videos of arbitrary length. First, it employs an autoregressive diffusion paradigm to achieve arbitrary-length video generation. To address the challenge of consistency, our key insight is to decouple the problem into inter-shot consistency and intra-shot coherence. We achieve this through a dual-level cache mechanism: a shot memory caches keyframes from preceding shots to maintain character and scene identity, while a temporal memory retains a history of frames from the current shot to ensure smooth, continuous motion. The proposed framework allows for flexible, multi-round user interaction to create multi-shot videos. Furthermore, due to this decoupled design, our method demonstrates high versatility by supporting downstream tasks such as multi-concept injection and video extension. To facilitate the training of our consistency-aware method, we also developed a comprehensive pipeline to construct a high-quality multi-shot video dataset. Extensive experimental results demonstrate that our method surpasses existing approaches on metrics for both consistency and aesthetic quality, opening up new possibilities for creating more consistent, controllable, and narrative-driven video content. Project Page: https://filmweaver.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
