arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.17342cs.CVcs.AI
收藏

面向点云协同表征与3D生成的掩码生成提取器

Masked Generative Extractor for Synergistic Representation and 3D Generation of Point Clouds

South China University of Technology(华南理工大学)

Hongliang Zeng, Ping Zhang, Fang Li, Jiahua Wang, Tingyu Ye, Pengteng Guo

AI总结:

针对现有点云表征方法无法真正生成3D形状的问题,提出Point-MGE框架,通过向量量化变分自编码器与滑动掩码率融合表征与生成学习,在多个下游任务取得SOTA并能生成高质量3D形状。

摘要:

表征学习与生成学习作为基于重建的方法,已在多个领域展现出相互促进的潜力。在点云处理领域,尽管现有研究已采用生成模型的训练策略来增强表征能力,但这些方法受限于无法真正生成3D形状。为探索深度融合3D表征学习与生成学习的优势,我们提出了名为Point-MGE(点云掩码生成提取器)的创新框架。具体而言,该框架首先利用向量量化变分自编码器重建3D形状的神经场表征,从而学习点云块的离散语义特征。随后,我们设计了滑动掩码率机制,以平滑从表征学习到生成学习的过渡。此外,我们的方法在学习高容量模型方面展现出强大的泛化能力,在多个下游任务上取得了新的最优性能。在形状分类任务中,Point-MGE在ModelNet40数据集上达到94.2%的准确率(提升1.0%),在ScanObjectNN数据集上达到92.9%的准确率(提升5.5%)。实验结果还证实,Point-MGE能够在无条件和条件设置下生成高质量的3D形状。

英文摘要:

Representation and generative learning, as reconstruction-based methods, have demonstrated their potential for mutual reinforcement across various domains. In the field of point cloud processing, although existing studies have adopted training strategies from generative models to enhance representational capabilities, these methods are limited by their inability to genuinely generate 3D shapes. To explore the benefits of deeply integrating 3D representation learning and generative learning, we propose an innovative framework called \textit{Point-MGE}. Specifically, this framework first utilizes a vector quantized variational autoencoder to reconstruct a neural field representation of 3D shapes, thereby learning discrete semantic features of point patches. Subsequently, we design a sliding masking ratios to smooth the transition from representation learning to generative learning. Moreover, our method demonstrates strong generalization capability in learning high-capacity models, achieving new state-of-the-art performance across multiple downstream tasks. In shape classification, Point-MGE achieved an accuracy of 94.2% (+1.0%) on the ModelNet40 dataset and 92.9% (+5.5%) on the ScanObjectNN dataset. Experimental results also confirmed that Point-MGE can generate high-quality 3D shapes in both unconditional and conditional settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑