arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.07587cs.CV
收藏

Let Occ Flow:自监督3D占用流预测

Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction

Zhejiang University(浙江大学);Huawei Cloud Computing Technologies Co., Ltd.(华为云计算技术有限公司)

Yili Liu, Linzhan Mou, Xuan Yu, Chenrui Han, Sitong Mao, Rong Xiong, Yue Wang

AI总结:

针对自动驾驶等的动态环境感知需求,提出首个仅用摄像头、无需3D标注的自监督3D占用与占用流联合预测方法Let Occ Flow,通过时序融合、可微渲染等技术,在主流数据集上取得优于现有SOTA的性能。

摘要:

动态环境的精准感知是自动驾驶与机器人系统的基础任务。本文提出Let Occ Flow,这是首个仅使用摄像头输入、无需3D标注即可联合预测3D占用与占用流的自监督研究。该方法采用TPV(三视图体素)实现统一场景表示,通过可变形注意力层进行特征聚合,引入全新的基于注意力的时序融合模块捕捉动态物体依赖关系,再经3D细化模块生成细粒度体素表示。此外,该方法将可微渲染拓展至3D体素流场,利用零样本2D分割与光流线索实现动态分解与运动优化。在nuScenes和KITTI数据集上的大量实验表明,该方法性能优于现有最先进方法,具备竞争力。项目主页为:https://eliliu2233.github.io/letoccflow/

英文摘要:

Accurate perception of the dynamic environment is a fundamental task for autonomous driving and robot systems. This paper introduces Let Occ Flow, the first self-supervised work for joint 3D occupancy and occupancy flow prediction using only camera inputs, eliminating the need for 3D annotations. Utilizing TPV for unified scene representation and deformable attention layers for feature aggregation, our approach incorporates a novel attention-based temporal fusion module to capture dynamic object dependencies, followed by a 3D refine module for fine-gained volumetric representation. Besides, our method extends differentiable rendering to 3D volumetric flow fields, leveraging zero-shot 2D segmentation and optical flow cues for dynamic decomposition and motion optimization. Extensive experiments on nuScenes and KITTI datasets demonstrate the competitive performance of our approach over prior state-of-the-art methods. Our project page is available at https://eliliu2233.github.io/letoccflow/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑