arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.00762cs.CV
收藏

Point Cloud Mamba:基于状态空间模型的点云学习

Point Cloud Mamba: Point Cloud Learning via State Space Model

Skywork AI(天工智联(Skywork AI))

Tao Zhang, Haobo Yuan, Lu Qi, Jiangning Zhang, Qianyu Zhou, Shunping Ji, Shuicheng Yan, Xiangtai Li

AI总结:

首次将Mamba应用于点云学习,提出Consistent Traverse Serialization、点提示和空间坐标映射位置编码,在多个数据集上超越PointNeXt等SOTA方法取得新最优性能。

摘要:

最近,状态空间模型展现出强大的全局建模能力和线性计算复杂度,与transformer形成对比。本研究致力于应用此类架构,以线性计算复杂度更高效、更有效地对点云数据进行全局建模。特别是,我们首次证明基于Mamba的点云方法能够超越以往基于transformer或多层感知机(MLPs)的方法。为使Mamba更有效地处理3D点云数据,我们提出一种新颖的Consistent Traverse Serialization方法,将点云转换为1D点序列,同时确保序列中的相邻点在空间上也相邻。Consistent Traverse Serialization通过排列x、y和z坐标的顺序产生六种变体,协同使用这些变体有助于Mamba全面观察点云数据。此外,为协助Mamba更有效地处理不同顺序的点序列,我们引入点提示以告知Mamba序列的排列规则。最后,我们提出基于空间坐标映射的位置编码,以更有效地将位置信息注入点云序列。Point Cloud Mamba超越了最先进的(SOTA)基于点的方法PointNeXt,并在ScanObjectNN、ModelNet40、ShapeNetPart和S3DIS数据集上取得新的SOTA性能。值得一提的是,当使用更强大的局部特征提取模块时,我们的PCM在S3DIS上取得79.6 mIoU,分别以5.5 mIoU和4.9 mIoU显著超越先前的SOTA模型DeLA和PTv3。

英文摘要:

Recently, state space models have exhibited strong global modeling capabilities and linear computational complexity in contrast to transformers. This research focuses on applying such architecture to more efficiently and effectively model point cloud data globally with linear computational complexity. In particular, for the first time, we demonstrate that Mamba-based point cloud methods can outperform previous methods based on transformer or multi-layer perceptrons (MLPs). To enable Mamba to process 3-D point cloud data more effectively, we propose a novel Consistent Traverse Serialization method to convert point clouds into 1-D point sequences while ensuring that neighboring points in the sequence are also spatially adjacent. Consistent Traverse Serialization yields six variants by permuting the order of \textit{x}, \textit{y}, and \textit{z} coordinates, and the synergistic use of these variants aids Mamba in comprehensively observing point cloud data. Furthermore, to assist Mamba in handling point sequences with different orders more effectively, we introduce point prompts to inform Mamba of the sequence's arrangement rules. Finally, we propose positional encoding based on spatial coordinate mapping to inject positional information into point cloud sequences more effectively. Point Cloud Mamba surpasses the state-of-the-art (SOTA) point-based method PointNeXt and achieves new SOTA performance on the ScanObjectNN, ModelNet40, ShapeNetPart, and S3DIS datasets. It is worth mentioning that when using a more powerful local feature extraction module, our PCM achieves 79.6 mIoU on S3DIS, significantly surpassing the previous SOTA models, DeLA and PTv3, by 5.5 mIoU and 4.9 mIoU, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑