超越平均值：基于高斯泼溅和嵌入袋的开放词汇3D场景理解
Beyond Averages: Open-Vocabulary 3D Scene Understanding with Gaussian Splatting and Bag of Embeddings
AI总结：
提出一种绕开可微渲染的3D场景理解新范式，利用预分解对象级高斯和多视图CLIP特征聚合构建“嵌入袋”，实现准确的开放词汇对象检索与无缝2D/3D任务适应。
摘要：
随着3D高斯泼溅（3DGS）的出现，新视角合成取得了显著进展，实现了实时照片级真实感渲染。然而，高斯泼溅固有的模糊性给3D场景理解带来了挑战，限制了其在AR/VR和机器人领域的更广泛应用。尽管最近的工作试图通过2D基础模型蒸馏来学习语义，但它们继承了根本的局限性：alpha混合会平均跨对象的语义，使得3D级别的理解无法实现。我们提出了一种范式转换的替代方案，完全绕开用于语义的可微渲染。我们的关键洞察是利用预分解的对象级高斯，并通过多视图CLIP特征聚合来表示每个对象，创建全面描述对象的“嵌入袋”。这允许：（1）通过比较文本查询与对象级（而非高斯级）嵌入来实现准确的开放词汇对象检索；（2）无缝任务适应：将对象ID传播到像素以进行2D分割，或传播到高斯以进行3D提取。实验表明，我们的方法有效克服了3D开放词汇对象提取的挑战，同时在2D开放词汇分割中保持与最先进性能相当的水平，确保了最小的折衷。
英文摘要：
Novel view synthesis has seen significant advancements with 3D Gaussian Splatting (3DGS), enabling real-time photorealistic rendering. However, the inherent fuzziness of Gaussian Splatting presents challenges for 3D scene understanding, restricting its broader applications in AR/VR and robotics. While recent works attempt to learn semantics via 2D foundation model distillation, they inherit fundamental limitations: alpha blending averages semantics across objects, making 3D-level understanding impossible. We propose a paradigm-shifting alternative that bypasses differentiable rendering for semantics entirely. Our key insight is to leverage predecomposed object-level Gaussians and represent each object through multiview CLIP feature aggregation, creating comprehensive "bags of embeddings" that holistically describe objects. This allows: (1) accurate open-vocabulary object retrieval by comparing text queries to object-level (not Gaussian-level) embeddings, and (2) seamless task adaptation: propagating object IDs to pixels for 2D segmentation or to Gaussians for 3D extraction. Experiments demonstrate that our method effectively overcomes the challenges of 3D open-vocabulary object extraction while remaining comparable to state-of-the-art performance in 2D open-vocabulary segmentation, ensuring minimal compromise.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
