arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.10135cs.CVcs.AI

BiWM:利用双向自回归推进开源交互式视频世界模型

BiWM: Advancing Open-Source Interactive Video World Models with Bidirectional Autoregression

LynnReal AI;Shanghai Innovation Institute(上海创新研究院);Shanghai Jiao Tong University(上海交通大学);Fudan University(复旦大学)

Shaohao Rui, Xiaofeng Mao, Zhanyu Zhang, Peijia Lin, Yansong Zhu, Yibo Zhang, Haibin Wan, Zhangrui Zhao, Weijie Ma

AI总结:

提出BiWM框架,通过双向自回归范式将预训练视频骨干转化为交互式世界模型,仅需两阶段训练(微调+分布匹配蒸馏),支持多尺度模型和长程生成,优于现有因果流水线。

摘要:

将双向视频扩散模型过渡到自回归范式提高了视频世界模型的交互性,但现有的因果流水线需要多个阶段(控制微调、自回归训练、因果初始化、少步蒸馏),并且由于误差累积,质量仍落后于双向模型。最近的世界模型如Yume-1.5和Matrix-Game-3.0采用双向自回归方法,通过自我纠正误差传播获得保真度和稳定的长程展开,但开源框架(如minWM)仅支持因果模型。我们提出BiWM,这是首个在双向自回归范式下用于交互式视频世界模型的全栈框架,联合优化生成质量和推理速度。从预训练视频骨干开始,BiWM通过微调注入相机控制,然后运行几步分布匹配蒸馏(DMD)阶段,将骨干转化为动作/相机可控的世界模型:仅需两个训练阶段(而非minWM的四个),在8xH200 GPU上几百步内收敛。单一方案覆盖Wan2.1-1.3B、Wan2.2-5B、HunyuanVideo-1.5-8B和LTX-2.3-22B,并支持现有双向模型的二次微调。BiWM实现了minWM失去可控性的真实相机控制,集成了可插拔历史压缩(FramePack风格和PackForcing风格)用于长程展开,并提供可选的NVFP4 4位训练/推理流水线。为对抗DMD的模式寻求退化,我们添加了GAN和覆盖前向KL目标,以保留场景动态。我们开源BiWM,用于资源受限的研究和高保真环境模拟。

英文摘要:

Interactive video world models commonly convert bidirectional video generators into causal autoregressive systems through control fine-tuning, autoregressive training, causal initialization, and few-step distillation. This pipeline is costly, while frozen causal histories accumulate errors that degrade long-horizon fidelity and controllability. We present BiWM, the first open-source full-stack training framework for bidirectional autoregressive video world models. BiWM retains full attention within each generated chunk and requires only two stages: camera/action-control fine-tuning and few-step Distribution Matching Distillation (DMD). Both stages converge within a few hundred optimizer steps on 8 H200 GPUs. The framework supports Wan2.1-T2V-1.3B, Wan2.2-TI2V-5B, HunyuanVideo-1.5-TI2V-8B, and LTX-2.3-22B, together with real-world camera control, pluggable long-history compression, and optional low-bit deployment. Supervised and forward-KL anchors mitigate DMD mode collapse and preserve scene dynamics. BiWM provides a compact, reproducible path from pretrained bidirectional video models to interactive, controllable, and efficient world models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑