PanoGS：用于3D开放词汇场景理解的基于高斯的全景分割
PanoGS: Gaussian-based Panoptic Segmentation for 3D Open Vocabulary Scene Understanding
AI总结：
针对现有3DGS开放词汇场景理解方法无法区分实例级信息的问题，提出PanoGS方法，通过金字塔三平面特征建模、语言引导图割与SAM辅助图聚类实现3D全景开放词汇理解，性能更具竞争力。
摘要：
近年来，3D Gaussian Splatting（3DGS，三维高斯泼溅）在开放词汇场景理解任务中展现出可观的性能。然而，现有方法无法区分3D实例级信息，通常仅预测场景特征与文本查询之间的热力图。本文提出PanoGS，一种新颖且有效的3D全景开放词汇场景理解方法。技术层面，为学习可扩展至大型室内场景的精准3D语言特征，我们采用金字塔三平面对潜在连续参数化特征空间建模，并使用3D特征解码器回归多视角融合的2D特征点云。此外，我们提出语言引导的图割方法，协同利用重建的几何结构与学习到的语言线索，将3D高斯基元分组为一组超基元。为获取3D一致的实例，我们执行基于图聚类的分割，结合SAM（Segment Anything Model，分割一切模型）引导的不同超基元间边缘亲和度计算。在广泛使用的数据集上开展的大量实验表明，该方法在3D全景开放词汇场景理解任务上取得了更优或更具竞争力的性能。项目页面：https://zju3dv.github.io/panogs。
英文摘要：
Recently, 3D Gaussian Splatting (3DGS) has shown encouraging performance for open vocabulary scene understanding tasks. However, previous methods cannot distinguish 3D instance-level information, which usually predicts a heatmap between the scene feature and text query. In this paper, we propose PanoGS, a novel and effective 3D panoptic open vocabulary scene understanding approach. Technically, to learn accurate 3D language features that can scale to large indoor scenarios, we adopt the pyramid tri-plane to model the latent continuous parametric feature space and use a 3D feature decoder to regress the multi-view fused 2D feature cloud. Besides, we propose language-guided graph cuts that synergistically leverage reconstructed geometry and learned language cues to group 3D Gaussian primitives into a set of super-primitives. To obtain 3D consistent instance, we perform graph clustering based segmentation with SAM-guided edge affinity computation between different super-primitives. Extensive experiments on widely used datasets show better or more competitive performance on 3D panoptic open vocabulary scene understanding. Project page: \href{https://zju3dv.github.io/panogs}{https://zju3dv.github.io/panogs}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
