ARCON：推进自动驾驶视频的自回归续写
Auto-Regressive Models Need Structural Registers: Semantic Foresight Improves Coherent Driving Video Continuation
AI总结：
本文提出ARCON，一种交替生成语义与RGB令牌的自回归视频续写方案，利用大型视觉模型学习结构化信息，并通过光流纹理拼接提升质量，在自动驾驶场景中实现长视频的一致生成。
摘要：
自回归大语言模型（LLMs）的最新进展已促使其应用于视频生成领域。本文探讨了利用大型视觉模型（LVMs）进行视频续写，这是构建世界模型和预测未来帧的关键任务。我们提出了ARCON方案，该方案在生成语义令牌和RGB令牌之间交替进行，使LVM能够显式学习高层级的结构化视频信息。我们发现，无需特殊设计，生成的RGB图像与语义图之间具有高度一致性。此外，我们采用了一种基于光流的纹理拼接方法来增强视觉质量。在自动驾驶场景中的实验表明，我们的模型能够一致地生成长视频。
英文摘要：
Front-view driving video continuation is a critical component for constructing sophisticated world models. However, maintaining long-term coherence faces the fundamental challenge of mitigating ''generative degeneration'' during the auto-regressive process. This phenomenon arises when the limited attention capacity of world models becomes saturated with high-entropy pixel details over extended sequences, leading to a collapse of global scene structure and motion intensity. To address this, we introduce STRIDE (STructural RegIsters for Decoupled Extrapolation), a large vision model architecture with an enforced semantic foresight mechanism. STRIDE decouples video continuation into two sub-tasks via interleaved generation of semantic and RGB tokens: forecasting low-entropy semantic maps, and subsequently generating high-frequency visual appearance conditioned on this structure. The semantic tokens explicitly function as ''structural registers'' -- dedicated memory slots that robustly retain the scene's long-term context and world dynamics. Furthermore, we employ an architecturally decoupled flow matching-based super-resolution module to enhance the final visual fidelity. Extensive experiments in autonomous driving scenarios demonstrate that STRIDE effectively mitigates generative degeneration, achieving minute-level, temporally consistent driving video continuation with strong generalization ability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
