OpenGaFF: 基于码本注意力的开放词汇高斯特征场
OpenGaFF: Open-Vocabulary Gaussian Feature Field with Codebook Attention
AI总结：
提出OpenGaFF框架，通过高斯特征场和码本注意力机制实现开放词汇3D场景理解，提升语义一致性和分割质量。
摘要：
理解基于高斯表示的开放词汇3D场景仍然具有挑战性，因为多视角观测下的语义预测碎片化且空间不一致。在本文中，我们提出了OpenGaFF，一个基于3D高斯泼溅构建的开放词汇3D场景理解新框架。我们方法的核心是一个高斯特征场，它将语义建模为高斯几何和外观的连续函数。通过显式地将语义预测条件于几何结构，该公式加强了几何与语义之间的耦合，从而在3D空间中相似结构上实现了更好的空间一致性。为了进一步强制执行对象级语义一致性，我们引入了一个结构化码本，作为一组共享的语义基元。此外，提出了一种码本引导的注意力机制，通过查询嵌入与学习到的码本条目之间的相似性匹配来检索语言特征，从而实现鲁棒的开放词汇推理，同时减少对象内特征方差。在标准2D和3D开放词汇基准上的大量实验表明，我们的方法持续优于先前的方法，实现了改进的分割质量、更强的3D语义一致性以及一个语义可解释的码本，为学习到的表示提供了洞察。
英文摘要：
Understanding open-vocabulary 3D scenes with Gaussian-based representations remains challenging due to fragmented and spatially inconsistent semantic predictions across multi-view observations. In this paper, we present OpenGaFF, a novel framework for open-vocabulary 3D scene understanding built upon 3D Gaussian Splatting. At the core of our method is a Gaussian Feature Field that models semantics as a continuous function of Gaussian geometry and appearance. By explicitly conditioning semantic predictions on geometric structure, this formulation strengthens the coupling between geometry and semantics, leading to improved spatial coherence across similar structures in 3D space. To further enforce object-level semantic consistency, we introduce a structured codebook that serves as a set of shared semantic primitives. Furthermore, a codebook-guided attention mechanism is proposed to retrieve language features via similarity matching between query embeddings and learned codebook entries, enabling robust open-vocabulary reasoning while reducing intra-object feature variance. Extensive experiments on standard 2D and 3D open-vocabulary benchmarks demonstrate that our method consistently outperforms prior approaches, achieving improved segmentation quality, stronger 3D semantic consistency and a semantically interpretable codebook that provides insight into the learned representation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
