arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.13772cs.CV

基于解耦动态流与图像辅助训练的高效占用世界模型

An Efficient Occupancy World Model via Decoupled Dynamic Flow and Image-assisted Training

FNii;SSE, CUHK-Shenzhen(香港中文大学(深圳)理工学院);HKU(香港大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室)

Haiming Zhang, Ying Xue, Xu Yan, Jiacheng Zhang, Weichao Qiu, Dongfeng Bai, Bingbing Liu, Shuguang Cui, Zhen Li

AI总结:

本文提出高效3D占用世界模型DFIT-OccWorld,通过解耦动态流进行体素扭曲并引入图像辅助训练范式,在大幅降低计算成本的同时实现了4D占用预测、运动规划和点云预测的最先进性能。

摘要:

自动驾驶领域正掀起对世界模型的研究热潮,旨在基于历史观测预测潜在的未来场景。本文提出了 DFIT-OccWorld,这是一种高效的3D占用世界模型,它利用解耦的动态流和图像辅助训练策略,大幅提升了4D场景预测性能。为简化训练过程,我们摒弃了以往的两阶段训练策略,创新性地将占用预测问题重新表述为解耦的体素扭曲过程。该模型通过体素流对现有观测进行扭曲来预测未来的动态体素,而静态体素则可通过位姿变换轻松获得。此外,该方法引入了图像辅助训练范式以增强预测的可靠性。具体而言,采用可微体渲染技术,通过预测的未来体素生成渲染深度图,并将其应用于基于渲染的光度一致性约束。实验证明了本方法的有效性,展示了其在 nuScenes 和 OpenScene 基准上的4D占用预测、端到端运动规划和点云预测任务中的最先进性能。具体而言,与现有的3D世界模型相比,它在实现最先进性能的同时,计算成本大幅降低。

英文摘要:

The field of autonomous driving is experiencing a surge of interest in world models, which aim to predict potential future scenarios based on historical observations. In this paper, we introduce DFIT-OccWorld, an efficient 3D occupancy world model that leverages decoupled dynamic flow and image-assisted training strategy, substantially improving 4D scene forecasting performance. To simplify the training process, we discard the previous two-stage training strategy and innovatively reformulate the occupancy forecasting problem as a decoupled voxels warping process. Our model forecasts future dynamic voxels by warping existing observations using voxel flow, whereas static voxels are easily obtained through pose transformation. Moreover, our method incorporates an image-assisted training paradigm to enhance prediction reliability. Specifically, differentiable volume rendering is adopted to generate rendered depth maps through predicted future volumes, which are adopted in render-based photometric consistency. Experiments demonstrate the effectiveness of our approach, showcasing its state-of-the-art performance on the nuScenes and OpenScene benchmarks for 4D occupancy forecasting, end-to-end motion planning and point cloud forecasting. Concretely, it achieves state-of-the-art performances compared to existing 3D world models while incurring substantially lower computational costs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑