arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.19212cs.CV
收藏

用于一致未来帧预测的解耦漂移扩散世界模型

Decoupled Drift-Diffusion World Models for Consistent Future Frame Prediction

University of Michigan(密歇根大学);NVIDIA(英伟达)

Anton Arapin, Aditya Singh, Yixin Wang, Jun Gao, Bernadette Bucher

AI总结:

该研究提出D3WM模型,通过联合回归确定性状态预测与空间不确定性图引导扩散过程,在机器人桌面操纵和相机运动两类动作空间的未来帧预测任务中性能优于基线。

摘要:

视觉信息的世界模型可在给定动作序列的条件下预测未来图像帧。这种预测的场景演化由确定性变化(如机器人平台上的受控运动)和随机变化(如被操纵物体或先前遮挡区域的渲染)共同驱动。我们的核心见解是,这些随机变化的空间位置本身是可学习的。我们利用这一见解,联合回归确定性下一状态预测和空间不确定性图,再用该不确定性图指导条件扩散过程。我们提出了解耦漂移扩散世界模型(Decoupled Drift-Diffusion World Model,D3WM),该模型使用经过高斯负对数似然目标训练的Transformer,生成确定性下一状态预测和异方差随机不确定性的空间图。学习到的不确定性图为条件扩散模型提供了结构化空间门:确定性区域锚定到预测状态,而不确定区域(如遮挡恢复区域或动态纹理)则由扩散过程采样。我们在方差门控随机微分方程框架下构建了这种分解,将确定性漂移与随机扩散分离开来。D3WM在两种不同类型的动作空间(机器人桌面操纵和相机运动)上,相较于具有竞争力的基线模型展现出了更优的结果。

英文摘要:

World models of visual information predict future image frames conditioned on given action sequences. This predicted scene evolution is driven by deterministic changes such as controlled motion on a robot platform, as well as stochastic changes such as rendering of manipulated objects or previously occluded regions. Our key insight is that the spatial location of these stochastic changes is itself learnable. We leverage this insight by jointly regressing both the deterministic next-state prediction and the spatial uncertainty map, then using the uncertainty map to guide a conditional diffusion process. We propose Decoupled Drift-Diffusion World Model (D3WM), which uses a transformer trained with a Gaussian negative-log-likelihood objective to produce both the deterministic next-state prediction and a spatial map of heteroscedastic aleatoric uncertainty. The learned uncertainty map then provides a structured spatial gate for a conditional diffusion model: deterministic regions are anchored to the predicted state, while uncertain regions, such as disocclusions or dynamic textures, are sampled by the diffusion process. We frame this decomposition under a variance-gated stochastic differential equation that separates deterministic drift from stochastic diffusion. D3WM shows improved results over competitive baselines in two distinct types of action spaces: robotic tabletop manipulation and camera motion.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑