稀疏编码提升用于高效3D语言高斯散射
Sparse Code Uplifting for Efficient 3D Language Gaussian Splatting
AI总结：
本文提出SCOUP方法，通过解耦语言表示学习与3D高斯优化，实现高效3D语义重建、存储和渲染。方法利用2D图像区域特征学习稀疏代码本表示，并通过加权稀疏聚合提升至3D高斯体，从而提升训练速度和内存效率。
摘要：
3D语言高斯散射（3DLGS）通过语言对齐的视觉特征增强3D高斯散射，以实现开放词汇3D场景理解。核心挑战是高效地将高维视觉-语言嵌入与数百万个3D高斯体关联，同时保持文本查询的高效特征渲染。现有方法要么直接在高斯体上存储密集特征，导致存储成本高且渲染慢，要么通过昂贵的每场景优化学习紧凑表示，需重复特征光栅化。没有现有方法同时实现快速3D语义重建、高效存储和快速渲染。我们提出SCOUP（稀疏编码提升），通过解耦语言表示学习与3D高斯优化，解决这三个问题。我们不直接在3D空间工作，而是完全利用与2D图像区域关联的特征学习基于稀疏代码本的表示，将每个区域与稀疏的代码本系数关联。然后通过加权稀疏聚合将这些系数提升到3D高斯体，其中每个高斯体在不同视角下累积代码本原子的系数。Top-K过滤提取每个高斯体的最主导多视角系数，实现高效存储和快速渲染。我们的方法在训练速度上达到最高400倍提升，同时在渲染速度下的内存效率比现有方法高3倍。在多个基准测试中，SCOUP在开放词汇查询精度上匹配或优于现有方法。
英文摘要：
3D Language Gaussian Splatting (3DLGS) augments 3D Gaussian Splatting with language-aligned visual features for open-vocabulary 3D scene understanding. A core challenge is efficiently associating high-dimensional vision-language embeddings with millions of 3D Gaussians while preserving efficient feature rendering for text-based querying. Existing methods either store dense features directly on Gaussians, causing high storage costs and slow rendering, or learn compact representations through expensive per-scene optimization with repeated feature rasterization. No existing method simultaneously achieves fast 3D semantic reconstruction, efficient storage, and fast rendering. We propose SCOUP (Sparse COde UPlifting), which addresses all three by decoupling language representation learning from 3D Gaussian optimization. Rather than working directly in 3D, we learn sparse codebook-based representations entirely using features associated with 2D image regions, associating each region with a sparse set of codebook coefficients. We then uplift these coefficients to 3D Gaussians with our weighted sparse aggregation using Gaussian-to-pixel associations, where each Gaussian accumulates coefficients over codebook atoms across views. Top-$K$ filtering then extracts the most dominant multi-view coefficients per Gaussian, enabling efficient storage and fast rendering. Our method achieves up to $400\times$ training speedup while being $3\times$ more memory efficient during training compared to the state-of-the-art in rendering speed. Across multiple benchmarks, SCOUP matches or outperforms existing methods in open-vocabulary querying accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
