高斯融合：迈向多模态3D高斯预训练
GaussFusion: Towards Multimodal 3D Gaussian Pretraining
AI总结：
研究提出GaussFusion多模态3D高斯预训练框架，通过跨模态语义对齐集成图像和文本监督，还提出高斯显著性引导的多尺度空洞掩码，实验表明该方法提高了高斯表示可迁移性，在ModelNet40和ScanObjectNN上有性能提升。
摘要：
3D高斯点云提供了一种联合建模几何和外观的显式表示，是3D表示学习的可扩展基础。现有的高斯表示预训练方法，如掩码高斯重建，主要捕获局部结构，但语义监督有限。本文提出了GaussFusion，一种用于3D高斯表示的多模态预训练框架。它通过跨模态语义对齐将图像和文本监督集成到掩码高斯建模中，使高斯编码器在预训练期间学习视觉和语言级别的语义信息。为更好地使掩码建模适应高斯基元的非均匀分布，还提出了高斯显著性引导的多尺度空洞掩码（GSHM）。GSHM基于高斯显著性构建空间连续的掩码区域，通过多尺度应用空洞掩码，鼓励编码器捕获细粒度局部模式和更广泛的结构依赖性。在下游任务上的大量实验表明，GaussFusion提高了高斯表示的可迁移性。特别是，GaussFusion在ModelNet40和ScanObjectNN（PB-T50-RS）上分别比Gaussian-MAE高出0.61%和3.85%。
英文摘要：
3D Gaussian Splatting provides an explicit representation that jointly models geometry and appearance, serving as a scalable foundation for 3D representation learning. Existing pre-training methods for Gaussian representations, such as masked Gaussian reconstruction, primarily capture local structures but offer limited semantic supervision. In this paper, we propose GaussFusion, a multimodal pre-training framework for 3D Gaussian representations. GaussFusion integrates image and text supervision into masked Gaussian modeling through cross-modal semantic alignment, enabling the Gaussian encoder to learn both visual and language-level semantic information during pre-training. To better adapt masked modeling to the non-uniform distribution of Gaussian primitives, we further propose Gaussian Salience-guided Multi-scale Hole Masking (GSHM). GSHM constructs spatially continuous masked regions based on Gaussian salience. By applying hole masks at multiple scales, GSHM encourages the encoder to capture both fine-grained local patterns and broader structural dependencies. Extensive experiments on downstream tasks demonstrate that GaussFusion improves the transferability of Gaussian representations. Notably, GaussFusion outperforms Gaussian-MAE on ModelNet40 and ScanObjectNN (PB-T50-RS) by 0.61\% and 3.85\%, respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
