发表机构
UCLA; Tsinghua University（加州大学洛杉矶分校; 清华大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
Stream4D用显式建模场景动力学的前馈4D重建奖励替代静态评判器，结合运动先验与感知锚，提升流式自回归扩散视频模型的4D重建质量、运动保留效果及人类对齐偏好。
AI 中文摘要
流式自回归扩散模型可实现实时长时序视频生成，但其训练目标优化的是局部帧预测，而非连贯世界的几何与动力学：长序列生成会累积几何漂移，退化为静态或不自然的运动。近期双向方法采用基于3D高斯溅射（3D Gaussian-Splatting）重建的奖励信号解决该问题，但单一刚性3D重建无法建模动态场景，因此该评判器会将真实物体运动惩罚为重建误差，且会因视频冻结而最大化这种误差，该捷径在自回归（AR）设置中尤其有害，因为每个数据块都会传播已静态的配置。本研究提出Stream4D，用前馈4D重建奖励替代静态评判器，该奖励显式建模场景动力学，使连贯运动获得高一致性奖励；为进一步引导运动幅度与质量，添加运动先验，奖励自然场景流幅度，同时惩罚抖动与非刚体伪影；最终方案将这两项与轻量感知锚结合。在多种自回归视频主干及不同生成时序下，Stream4D提升了4D重建质量，更有效保留运动，且获得更高的人类对齐偏好。项目页面：this https URL
英文摘要
Streaming autoregressive diffusion models enable real-time, long-horizon video generation, but their training objectives optimize local frame prediction rather than the geometry and dynamics of a coherent world: long rollouts accumulate geometric drift and degrade into static or unnatural motion. Recent bidirectional approaches address this problem using rewards signals built upon 3D Gaussian-Splatting reconstruction. However, a single rigid 3d reconstruction cannot model a dynamic scene, so this critic penalizes genuine object motion as reconstruction error and is maximized by freezing the video. This shortcut is especially detrimental in the AR setting, where each chunk can propagate an already-static configuration. In this work, we propose Stream4D, which replaces the static critic with a feed-forward 4D reconstruction reward that explicitly models scene dynamics, allowing coherent motion to receive high consistency rewards. To further guide motion magnitude and quality, we add a motion prior that rewards natural scene-flow magnitude while penalizing jitter and non-rigid artifacts. Our final recipe combines these two terms with a lightweight perceptual anchor. Across various autoregressive video backbones and various generation horizons, Stream4D improves 4D reconstruction quality, preserves motion more effectively, and achieves higher human-aligned preference. Project page: https://banyuanhao.github.io/Stream4D/
