arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.04034cs.CV
收藏

GaussianGraph:面向开放世界场景理解的基于3D Gaussian的场景图生成

GaussianGraph: 3D Gaussian-based Scene Graph Generation for Open-world Scene Understanding

School of Automation, Beijing Institute of Technology(北京理工大学自动化学院);National Key Lab of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology(北京理工大学自主智能无人系统全国重点实验室)

Xihan Wang, Dianyi Yang, Yu Gao, Yufeng Yue, Yi Yang, Mengyin Fu

AI总结:

针对3DGS方法分割精度低且缺乏空间推理能力的问题,提出GaussianGraph框架,通过自适应语义聚类与场景图生成整合物体属性与空间关系,显著提升了语义分割与物体定位性能。

摘要:

3D Gaussian Splatting(3DGS)的最新进展显著提升了语义场景理解能力,使自然语言查询能够定位场景内的物体。然而,现有方法主要侧重于将压缩的CLIP特征嵌入到3D Gaussians中,存在物体分割精度低和缺乏空间推理能力的问题。为了解决这些局限性,我们提出了GaussianGraph,这是一个通过整合自适应语义聚类和场景图生成来增强基于3DGS的场景理解的新颖框架。我们引入了一种“Control-Follow”聚类策略,该策略动态适应场景尺度和特征分布,避免了特征压缩并显著提升了分割精度。此外,我们通过整合从2D foundation models中提取的物体属性和空间关系来丰富场景表示。为了解决空间关系中的不准确问题,我们提出了3D校正模块,通过空间一致性验证过滤不合理的关系,确保可靠的场景图构建。在三个数据集上的大量实验表明,GaussianGraph在语义分割和物体定位任务中均优于现有最先进方法,为复杂场景理解和交互提供了稳健的解决方案。

英文摘要:

Recent advancements in 3D Gaussian Splatting(3DGS) have significantly improved semantic scene understanding, enabling natural language queries to localize objects within a scene. However, existing methods primarily focus on embedding compressed CLIP features to 3D Gaussians, suffering from low object segmentation accuracy and lack spatial reasoning capabilities. To address these limitations, we propose GaussianGraph, a novel framework that enhances 3DGS-based scene understanding by integrating adaptive semantic clustering and scene graph generation. We introduce a "Control-Follow" clustering strategy, which dynamically adapts to scene scale and feature distribution, avoiding feature compression and significantly improving segmentation accuracy. Additionally, we enrich scene representation by integrating object attributes and spatial relations extracted from 2D foundation models. To address inaccuracies in spatial relationships, we propose 3D correction modules that filter implausible relations through spatial consistency verification, ensuring reliable scene graph construction. Extensive experiments on three datasets demonstrate that GaussianGraph outperforms state-of-the-art methods in both semantic segmentation and object grounding tasks, providing a robust solution for complex scene understanding and interaction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑