ShapeSplat：大规模高斯溅射数据集及其自监督预训练
ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretraining
AI总结：
本文构建大规模3D高斯溅射数据集ShapeSplat，并提出Gaussian-MAE方法，通过自监督预训练与微调，揭示了质心分布变化对分类与分割的影响，并引入特征分组与池化层提升性能。
摘要：
三维高斯溅射（3DGS）已成为许多视觉任务中三维表示的事实标准方法。这要求直接在表示空间中进行三维理解。为促进该方向的研究，我们首先利用常用的ShapeNet、ModelNet和Objaverse数据集构建了大规模3DGS数据集ShapeSplat。我们的数据集ShapeSplat包含206K个对象，涵盖87个独特类别，其标签与各自数据集一致。该数据集的创建使用了相当于在TITAN XP GPU上3.8 GPU年的计算资源。我们利用该数据集进行无监督预训练和监督微调，用于分类和分割任务。为此，我们引入了Gaussian-MAE，其突显了从高斯参数进行表示学习的独特优势。通过大量实验，我们提供了若干有价值的见解。特别地，我们表明：（1）优化后的高斯溅射质心分布与用于初始化的均匀采样点云对应分布显著不同；（2）当仅使用质心时，这种分布变化导致分类性能下降，但分割任务性能提升；（3）为利用额外的高斯参数，我们提出了在归一化特征空间中的高斯特征分组，以及溅射池化层，提供了一种定制解决方案，以有效分组和嵌入相似的高斯，从而在微调任务中带来显著改进。
英文摘要：
3D Gaussian Splatting (3DGS) has become the de facto method of 3D representation in many vision tasks. This calls for the 3D understanding directly in this representation space. To facilitate the research in this direction, we first build ShapeSplat, a large-scale dataset of 3DGS using the commonly used ShapeNet, ModelNet and Objaverse datasets. Our dataset ShapeSplat consists of 206K objects spanning over 87 unique categories, whose labels are in accordance with the respective datasets. The creation of this dataset utilized the compute equivalent of 3.8 GPU years on a TITAN XP GPU. We utilize our dataset for unsupervised pretraining and supervised finetuning for classification and segmentation tasks. To this end, we introduce Gaussian-MAE, which highlights the unique benefits of representation learning from Gaussian parameters. Through exhaustive experiments, we provide several valuable insights. In particular, we show that (1) the distribution of the optimized GS centroids significantly differs from the uniformly sampled point cloud (used for initialization) counterpart; (2) this change in distribution results in degradation in classification but improvement in segmentation tasks when using only the centroids; (3) to leverage additional Gaussian parameters, we propose Gaussian feature grouping in a normalized feature space, along with splats pooling layer, offering a tailored solution to effectively group and embed similar Gaussians, which leads to notable improvement in finetuning tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
