arXiv 2407.073742024-07-10cs.CV

DuInNet：面向点云补全的双模态特征交互

DuInNet: Dual-Modality Feature Interaction for Point Cloud Completion

College of Electronic Science and Technology, National University of Defense Technology (NUDT)（国防科技大学电子科学学院）；School of Surveying and Mapping, Information Engineering University（信息工程大学测绘学院）

Xinpu Liu, Baolin Hou, Hanyun Wang, Ke Xu, Jianwei Wan, Yulan Guo

备注：: Under Review, 13 pages, 7 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文构建了含40类近40万对数据的多模态点云补全基准ModelNet-MPC，提出双模态特征交互网络DuInNet，通过迭代交互特征与自适应分块生成适配多任务，在多项补全任务上性能优于现有最优方法。

摘要：

为进一步推动多模态点云补全的发展，本文构建了一个大规模多模态点云补全基准数据集ModelNet-MPC，其具备更丰富的形状类别与更多样的测试数据，包含40个类别的近40万对高质量点云与渲染图像。除全监督点云补全任务外，ModelNet-MPC还设置了去噪补全和零样本学习补全两项额外任务，用于模拟真实场景并验证现有方法对噪声的鲁棒性及跨类别迁移能力。同时，针对现有多模态补全流程通常采用单向融合机制、忽略图像模态中蕴含的形状先验的问题，本文提出双模态特征交互网络（Dual-Modality Feature Interaction Network，DuInNet）。DuInNet通过双特征交互模块迭代交互点云与图像之间的特征，同时学习形状的几何与纹理特征。为适配全监督、去噪、零样本学习点云补全等特定任务，本文提出自适应点生成器，以分块方式生成完整点云，并为两种模态分配不同权重。在ShapeNet-ViPC和ModelNet-MPC基准上开展的大量实验表明，DuInNet在所有补全任务中均展现出优于现有最优方法的性能、鲁棒性与迁移能力。代码与数据集即将公开。

英文摘要：

To further promote the development of multimodal point cloud completion, we contribute a large-scale multimodal point cloud completion benchmark ModelNet-MPC with richer shape categories and more diverse test data, which contains nearly 400,000 pairs of high-quality point clouds and rendered images of 40 categories. Besides the fully supervised point cloud completion task, two additional tasks including denoising completion and zero-shot learning completion are proposed in ModelNet-MPC, to simulate real-world scenarios and verify the robustness to noise and the transfer ability across categories of current methods. Meanwhile, considering that existing multimodal completion pipelines usually adopt a unidirectional fusion mechanism and ignore the shape prior contained in the image modality, we propose a Dual-Modality Feature Interaction Network (DuInNet) in this paper. DuInNet iteratively interacts features between point clouds and images to learn both geometric and texture characteristics of shapes with the dual feature interactor. To adapt to specific tasks such as fully supervised, denoising, and zero-shot learning point cloud completions, an adaptive point generator is proposed to generate complete point clouds in blocks with different weights for these two modalities. Extensive experiments on the ShapeNet-ViPC and ModelNet-MPC benchmarks demonstrate that DuInNet exhibits superiority, robustness and transfer ability in all completion tasks over state-of-the-art methods. The code and dataset will be available soon.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。