arXiv 2605.045062026-05-13cs.CVcs.AI

Ilov3Splat：基于高斯散射的实例级开放词汇3D场景理解

Ilov3Splat: Instance-Level Open-Vocabulary 3D Scene Understanding in Gaussian Splatting

School of Electrical Engineering and Robotics（电气工程与机器人学学院）；Queensland University of Technology（昆士兰理工大学）；CSIRO Robotics（CSIRO机器人部）；CSIRO

Binh Long Nguyen, Kien Nguyen, Sridha Sridharan, Clinton Fookes, Peyman Moghadam

备注：: The International Conference on Pattern Recognition (ICPR) 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Ilov3Splat框架，通过增强高斯散射以实现实例级开放词汇3D场景理解，利用多分辨率哈希嵌入和对比损失提升语言语义关联与实例区分能力。

摘要：

我们介绍了Ilov3Splat，一种基于3D高斯散射（3D-GS）的新型框架，用于实例级开放词汇3D场景理解。大多数先前工作依赖于基于2D渲染的匹配或点级语义关联，这会破坏跨视角一致性，缺乏连贯的实例级推理，并限制下游3D任务的精度。为了解决这些限制，我们的方法通过增强高斯散射以视图一致的特征场，联合优化场景几何和语义表示。具体而言，我们利用多分辨率哈希嵌入高效编码语言对齐的CLIP特征，实现3D空间中的密集且连贯的语言定位。我们进一步通过SAM掩码上的对比损失训练实例特征场，支持跨视角的细粒度物体区分。在推理时，CLIP编码的查询与学习到的特征匹配，随后通过两阶段3D聚类检索相关高斯组。这使我们的框架能够基于自然语言描述识别3D场景中的任意物体，而无需类别监督或手动标注。在标准基准测试中，Ilov3Splat在物体选择和实例分割方面均优于先前的开放词汇3D-GS方法，提供了一种灵活且准确的语言驱动3D场景理解解决方案。项目页面：https://csiro-robotics.github.io/Ilov3Splat。

英文摘要：

We introduce Ilov3Splat, a novel framework for instance-level open-vocabulary 3D scene understanding built on 3D Gaussian Splatting (3D-GS). Most prior work depends on 2D rendering-based matching or point-level semantic association, which undermines cross-view consistency, lacks coherent instance-level reasoning, and limits precision in downstream 3D tasks. To address these limitations, our method jointly optimizes scene geometry and semantic representations by augmenting Gaussian splats with view-consistent feature fields. Specifically, we leverage multi-resolution hash embedding to efficiently encode language-aligned CLIP features, enabling dense and coherent language grounding in 3D space. We further train an instance feature field using contrastive loss over SAM masks, supporting fine-grained object distinction across views. At inference time, CLIP-encoded queries are matched against the learned features, followed by two-stage 3D clustering to retrieve relevant Gaussian groups. This enables our framework to identify arbitrary objects in 3D scenes based on natural language descriptions, without requiring category supervision or manual annotations. Experiments on standard benchmarks demonstrate that Ilov3Splat outperforms prior open-vocabulary 3D-GS methods in both object selection and instance segmentation, offering a flexible and accurate solution for language-driven 3D scene understanding. Project page: https://csiro-robotics.github.io/Ilov3Splat.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。