arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.08992cs.CV

用于3D目标检测的跨模态Mamba与渐进式解码器双域同质融合

Dual-Domain Homogeneous Fusion with Cross-Modal Mamba and Progressive Decoder for 3D Object Detection

Xuzhong Hu, Zaipeng Duan, Pei An, Jun zhang, Jie Ma

AI总结:

针对BEV同质融合的压缩、不对齐及稠密体素融合成本问题,提出DDHFusion,结合稀疏体素与BEV双域、跨模态Mamba、PQG和渐进式解码器以提升3D检测精度。

摘要:

在同质BEV域中融合LiDAR与图像特征已成为自动驾驶3D目标检测的流行方法。然而,该范式受到过度特征压缩的限制。尽管一些工作探索稠密体素融合以实现更好的特征交互,但它们面临高昂的计算成本和查询生成方面的挑战。此外,两个域中的特征不对齐会导致检测精度欠佳。为解决这些局限,我们提出双域同质融合网络(DDHFusion),在利用BEV域和体素域互补性的同时缓解其缺陷。具体而言,我们首先使用lift-splat-shot以及所提出的语义感知特征采样(SAFS)模块,将图像特征转换为BEV和稀疏体素表示;后者通过丢弃不重要的体素显著降低计算开销。接下来,我们引入同质体素与BEV融合(HVF和HBF)网络,用于各自域内的多模态融合。它们配备新颖的跨模态Mamba块,以解决特征不对齐并实现全面的场景感知。输出的体素特征被注入BEV空间,以补偿直接高度压缩带来的信息损失。在查询选择期间,在BEV域中实施渐进式查询生成(PQG)机制,以减少由特征压缩导致的假阴性。此外,我们提出渐进式解码器(QD),它利用可变形注意力和多模态体素特征混合(MMVFM)块,依次聚合不仅上下文丰富的BEV特征,还有几何感知的体素特征,以实现精确分类和边界框回归。

英文摘要:

Fusing LiDAR and image features in a homogeneous BEV domain has become popular for 3D object detection in autonomous driving. However, this paradigm is constrained by the excessive feature compression. While some works explore dense voxel fusion to enable better feature interaction, they face high computational costs and challenges in query generation. Additionally, feature misalignment in both domains results in suboptimal detection accuracy. To address these limitations, we propose a Dual-Domain Homogeneous Fusion network (DDHFusion), which leverages the complementarily of both BEV and voxel domains while mitigating their drawbacks. Specifically, we first transform image features into BEV and sparse voxel representations using lift-splat-shot and our proposed Semantic-Aware Feature Sampling (SAFS) module. The latter significantly reduces computational overhead by discarding unimportant voxels. Next, we introduce Homogeneous Voxel and BEV Fusion (HVF and HBF) networks for multi-modal fusion within respective domains. They are equipped with novel cross-modal Mamba blocks to resolve feature misalignment and enable comprehensive scene perception. The output voxel features are injected into the BEV space to compensate for the information loss brought by direct height compression. During query selection, the Progressive Query Generation (PQG) mechanism is implemented in the BEV domain to reduce false negatives caused by feature compression. Furthermore, we propose a Progressive Decoder (QD) that sequentially aggregates not only context-rich BEV features but also geometry-aware voxel features with deformable attention and the Multi-Modal Voxel Feature Mixing (MMVFM) block for precise classification and box regression.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑