CausalCine:多镜头视频叙事的实时自回归生成
CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narratives
AI总结:
CausalCine通过自回归框架实现多镜头视频生成,通过动态提示和记忆路由保持跨镜头连贯性,优于现有自回归模型,实现实时交互生成。
摘要:
自回归视频生成旨在实现实时、开放式的合成。然而,电影叙事不仅仅是单个场景的无限延伸,需要通过不断演变的事件、视角变化和离散镜头边界推进。现有自回归模型在这一设置中表现不佳。训练主要针对短时间延续,将长序列视为扩展的单个镜头,导致长时间生成时出现运动停滞和语义漂移。为弥合这一差距,我们引入CausalCine,一种交互式自回归框架,将多镜头视频生成转化为在线导演过程。CausalCine在镜头变化时进行因果生成,接受动态提示,并在不重生成先前镜头的情况下重用上下文。为此,我们首先在原生多镜头序列上训练因果基础模型,学习复杂的镜头过渡,以加速。然后提出内容感知记忆路由(CAMR),根据注意力相关性评分动态检索历史KV条目,而非时间接近性,从而在有限的活动内存下保持跨镜头一致性。最后,我们将因果基础模型蒸馏成一个几步生成器,用于实时交互生成。大量实验表明,CausalCine显著优于自回归基线,并接近双向模型的能力,同时解锁因果生成的流式交互性。演示可访问https://yihao-meng.github.io/CausalCine/
英文摘要:
Autoregressive video generation aims at real-time, open-ended synthesis. Yet, cinematic storytelling is not merely the endless extension of a single scene; it requires progressing through evolving events, viewpoint shifts, and discrete shot boundaries. Existing autoregressive models often struggle in this setting. Trained primarily for short-horizon continuation, they treat long sequences as extended single shots, inevitably suffering from motion stagnation and semantic drift during long rollouts. To bridge this gap, we introduce CausalCine, an interactive autoregressive framework that transforms multi-shot video generation into an online directing process. CausalCine generates causally across shot changes, accepts dynamic prompts on the fly, and reuses context without regenerating previous shots. To achieve this, we first train a causal base model on native multi-shot sequences to learn complex shot transitions prior to acceleration. We then propose Content-Aware Memory Routing (CAMR), which dynamically retrieves historical KV entries according to attention-based relevance scores rather than temporal proximity, preserving cross-shot coherence under bounded active memory. Finally, we distill the causal base model into a few-step generator for real-time interactive generation. Extensive experiments demonstrate that CausalCine significantly outperforms autoregressive baselines and approaches the capability of bidirectional models while unlocking the streaming interactivity of causal generation. Demo available at https://yihao-meng.github.io/CausalCine/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。