为实时自回归视频扩散适应VACE
Adapting VACE for Real-Time Autoregressive Video Diffusion
AI总结：
VACE被适配用于实时自回归视频生成，通过调整参考帧路径以兼容因果注意力，保留固定分块大小和KV缓存，同时增加20-30%的延迟开销，但显存成本低，参考到视频的保真度下降。
摘要：
我们描述了VACE（视频一体化创建与编辑）为实时自回归视频生成的适应。VACE提供了统一的视频控制（参考引导、结构条件化、修复和时间扩展），但假设全序列的双向注意力，使其与需要固定分块大小和因果注意力的流式管道不兼容。关键的修改是将参考帧从扩散潜在空间移动到并行的条件路径中，保留自回归模型所需的固定分块大小和KV缓存。这种适应可以重用已预训练的VACE权重而无需额外训练。在13亿和140亿模型规模上，VACE为结构控制和修复增加了20-30%的延迟开销，与基础模型相比，显存成本可忽略不计。由于因果注意力限制，参考到视频的保真度严重下降，相比批量VACE。参考实现可在https://github.com/daydreamlive/scope上获得。
英文摘要：
We describe an adaptation of VACE (Video All-in-one Creation and Editing) for real-time autoregressive video generation. VACE provides unified video control (reference guidance, structural conditioning, inpainting, and temporal extension) but assumes bidirectional attention over full sequences, making it incompatible with streaming pipelines that require fixed chunk sizes and causal attention. The key modification moves reference frames from the diffusion latent space into a parallel conditioning pathway, preserving the fixed chunk sizes and KV caching that autoregressive models require. This adaptation reuses existing pretrained VACE weights without additional training. Across 1.3B and 14B model scales, VACE adds 20-30% latency overhead for structural control and inpainting, with negligible VRAM cost relative to the base model. Reference-to-video fidelity is severely degraded compared to batch VACE due to causal attention constraints. A reference implementation is available at https://github.com/daydreamlive/scope.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
