Directing the World: 具有组合式人体-相机控制的快速自回归视频生成
Directing the World: Fast Autoregressive Video Generation with Compositional Human-Camera Control
AI总结：
提出一种快速自回归框架，通过解耦控制学习并保持统一视频先验，实现人体运动和相机轨迹的组合控制，支持稳定长程生成与高视觉质量。
摘要：
构建交互式世界模型需要在保持长程可控动态的同时生成逼真视频。自回归视频生成提供了可扩展的基础，但在扩展生成过程中存在误差累积和时间退化问题。在异构控制（如人体运动和相机轨迹）下，这一问题进一步加剧，这些控制可能干扰并破坏预训练的视频先验，而现有方法往往在可控性和视觉质量之间进行权衡。我们提出“Directing the World”，一种用于可控世界模型视频生成的快速自回归框架，具有组合式人体运动和相机轨迹控制。我们的核心思想是在保持统一的自回归视频先验的同时解耦控制学习。我们引入了一种快慢记忆训练策略，以稳定长程生成学习并提高收敛性。对于人体运动控制，我们设计了一种t引导的动态投影机制和一种改进的Motion-CFG策略，实现了时间平滑且准确的运动对齐，而不降低视觉保真度，并支持多人场景。通过学习鲁棒的运动先验，我们引入了一个第二阶段相机轨迹控制模块，将人体动态与视角变化组合起来，实现连贯的世界探索。我们还构建了一个大规模数据集，包含同步的视频、文本、人体运动和相机轨迹标注，并分为运动中心和相机中心子集以进行解耦训练。大量实验表明，该方法实现了稳定的长程生成、精确的可控性和高视觉质量。更多信息请访问此链接。
英文摘要：
Building interactive world models requires generating realistic videos while maintaining controllable dynamics over long horizons. Autoregressive video generation offers a scalable foundation, but suffers from error accumulation and temporal degradation during extended rollouts. This issue is further amplified under heterogeneous controls such as human motion and camera trajectories, which may interfere and destabilize a pretrained video prior, while existing methods often trade off controllability and visual quality. We propose "Directing the World", a fast autoregressive framework for controllable world-model video generation with compositional human-motion and camera-trajectory control. Our key idea is to decouple control learning while preserving a unified autoregressive video prior. We introduce a Fast-Slow Memory training strategy to stabilize long-horizon rollout learning and improve convergence. For human motion control, we design a t-guided Dynamic Projection mechanism and a refined Motion-CFG strategy, enabling temporally smooth and accurate motion alignment without degrading visual fidelity, and supporting multi-person control.After learning a robust motion prior, we introduce a second-stage camera-trajectory control module to compose human dynamics with viewpoint changes for coherent world exploration. We further construct a large-scale dataset with synchronized video, text, human-motion, and camera-trajectory annotations, organized into motion-centric and camera-centric subsets for decoupled training. Extensive experiments show stable long-horizon generation with precise controllability and high visual quality. See more at https://whydahuzi.github.io/Directing-the-World.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
