发表机构
NVIDIA; UC Berkeley; UCLA（英伟达; 加州大学伯克利分校; 加州大学洛杉矶分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对世界动作模型训练中动作噪声与视频噪声不匹配的问题，提出CtrlWAM，通过模拟器执行扰动动作并配对带噪视觉后果，引入扭曲视频-动作噪声调度和多智能体接口，提升动作预测、命令遵循与可控性。
AI 中文摘要
世界动作模型（WAM）联合预测动作（意图）和视觉未来（预见）。标准训练会同时向记录的动作和视频添加噪声，但此类训练范式引入了不匹配：受扰动的动作意味着反事实的未来视觉，而带噪视频仍与真实记录（GT）绑定。在低噪声情况下，尽管添加了噪声，场景几何甚至动态行为仍可从带噪的未来帧中清晰可见。我们提出CtrlWAM，它在模拟器中执行受扰动的动作，并将其与带噪的视觉后果配对，以进行联合WAM学习。为了适应视频和动作不同的去噪需求，我们引入了扭曲视频-动作噪声调度，旨在随着动作预测的演变保持视觉布局的响应性。我们进一步将动作接口从仅自我控制扩展到可变数量的智能体流，使统一模型能够表示多个智能体的预测或命令未来。驾驶实验表明，动作预测更准确，生成的视频与动作之间的一致性更强，并且对提供命令的遵循更好；机器人实验表明，运动保真度和可控性更强。匹配对照支持离路径渲染对命令遵循和操作保真度的益处。总之，这些发现有助于构建更可控的世界动作模型。项目页面：此https URL
英文摘要
World action models (WAMs) jointly predict actions (intent) and visual future (foresight). Standard training adds noise to recorded actions and video simultaneously, but such training paradigms introduce a mismatch: perturbed actions imply counterfactual future visual, while the noised video remains tied to the GT recording. In low-noise regime, the scene geometry and even the dynamic behavior remain clearly visible from the noisy future frames despite the added noise. We present CtrlWAM, which executes perturbed actions in a simulator and pairs them with their noised visual consequences for joint WAM learning. To accommodate the different denoising requirements of video and actions, we introduce warped video--action noise schedules that aim to keep visual layout responsive as action predictions evolve. We further extend the action interface from ego-only control to a variable number of agent streams, allowing a unified model to represent predicted or commanded futures for multiple agents. Driving experiments show more accurate action forecasts, closer agreement between generated video and actions, and better following of supplied commands; robotics experiments show stronger motion fidelity and controllability. Matched controls support the benefit of off-path renders for command following and manipulation fidelity. Together, these findings contribute to a more controllable world action model. Project page: https://ctrl-wam.github.io/
CommentsProject page: https://ctrl-wam.github.io/
