GaussianGraph: 3D Gaussian-based Scene Graph Generation for Open-world Scene Understanding
School of Automation, Beijing Institute of Technology(北京理工大学自动化学院);National Key Lab of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology(北京理工大学自主智能无人系统全国重点实验室)
Xihan Wang, Dianyi Yang, Yu Gao, Yufeng Yue, Yi Yang, Mengyin Fu
3D Gaussian Splatting(3DGS)的最新进展显著提升了语义场景理解能力,使自然语言查询能够定位场景内的物体。然而,现有方法主要侧重于将压缩的CLIP特征嵌入到3D Gaussians中,存在物体分割精度低和缺乏空间推理能力的问题。为了解决这些局限性,我们提出了GaussianGraph,这是一个通过整合自适应语义聚类和场景图生成来增强基于3DGS的场景理解的新颖框架。我们引入了一种“Control-Follow”聚类策略,该策略动态适应场景尺度和特征分布,避免了特征压缩并显著提升了分割精度。此外,我们通过整合从2D foundation models中提取的物体属性和空间关系来丰富场景表示。为了解决空间关系中的不准确问题,我们提出了3D校正模块,通过空间一致性验证过滤不合理的关系,确保可靠的场景图构建。在三个数据集上的大量实验表明,GaussianGraph在语义分割和物体定位任务中均优于现有最先进方法,为复杂场景理解和交互提供了稳健的解决方案。
英文摘要:
Recent advancements in 3D Gaussian Splatting(3DGS) have significantly improved semantic scene understanding, enabling natural language queries to localize objects within a scene. However, existing methods primarily focus on embedding compressed CLIP features to 3D Gaussians, suffering from low object segmentation accuracy and lack spatial reasoning capabilities. To address these limitations, we propose GaussianGraph, a novel framework that enhances 3DGS-based scene understanding by integrating adaptive semantic clustering and scene graph generation. We introduce a "Control-Follow" clustering strategy, which dynamically adapts to scene scale and feature distribution, avoiding feature compression and significantly improving segmentation accuracy. Additionally, we enrich scene representation by integrating object attributes and spatial relations extracted from 2D foundation models. To address inaccuracies in spatial relationships, we propose 3D correction modules that filter implausible relations through spatial consistency verification, ensuring reliable scene graph construction. Extensive experiments on three datasets demonstrate that GaussianGraph outperforms state-of-the-art methods in both semantic segmentation and object grounding tasks, providing a robust solution for complex scene understanding and interaction.
Holistic Large-Scale Scene Reconstruction via Mixed Gaussian Splatting
School of Computer Science, Wuhan University(武汉大学计算机学院);School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院);School of Electronic Information, Wuhan University(武汉大学电子信息学院);State Key Lab. of LIESMARS, Wuhan University(武汉大学LIESMARS国家重点实验室);Institute for Math & AI, Wuhan University(武汉大学数学与人工智能研究院)
3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian Splatting
The Chinese University of Hong Kong(香港中文大学);Stanford University(斯坦福大学);Snap Inc.(Snap公司);University of California Los Angeles(加利福尼亚大学洛杉矶分校);ByteDance(字节跳动)
HGS-Mapping: Online Dense Mapping Using Hybrid Gaussian Representation in Urban Scenes
Fudan University(复旦大学);School of Future Technology, Harbin Institute of Technology(哈尔滨工业大学未来技术学院);Department of Computer Science and Engineering, Shanghai Jiao Tong University(上海交通大学计算机科学与工程系)
SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);ETH Zürich(苏黎世联邦理工学院);University of Amsterdam(阿姆斯特丹大学);Johns Hopkins University(约翰·霍普金斯大学);University of Pisa(比萨大学);University of Trento(特伦托大学)
D-MiSo: Editing Dynamic 3D Scenes using Multi-Gaussians Soup
Doctoral School of Exact and Natural Sciences(精确与自然科学博士学院);Jagiellonian University(雅盖隆大学);Warsaw University of Technology(华沙工业大学);Sano Centre for Computational Medicine(萨诺计算医学中心);University of Cambridge(剑桥大学);IDEAS NCBR
DriveSplat: Unified Neural Gaussian Reconstruction for Dynamic Driving Scenes
State Key Laboratory of Multimodal Artificial Intelligence Systems(多模态人工智能系统国家重点实验室);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Zhongguancun Academy(中关村学院);Waytous
SparseStreet: Sparse Gaussian Splatting for Real-Time Street Scene Simulation
Peking University(北京大学);Chinese Academy of Sciences(中国科学院);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Autonomous Driving Development, NIO(蔚来自动驾驶开发)
RelayGS: Reconstructing Dynamic Scenes with Large-Scale and Complex Motions via Relay Gaussians
School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Peng Cheng Laboratory(鹏城实验室);Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology(广东省超高清沉浸式媒体技术重点实验室);Peking University Shenzhen Graduate School(北京大学深圳研究生院)
STD-GS: Exploring Frame-Event Interaction for SpatioTemporal-Disentangled Gaussian Splatting to Reconstruct High-Dynamic Scene
National Key Lab of Multispectral Information Intelligent Processing Technology(多谱信息智能处理国家级重点实验室);School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(人工智能与自动化学院,华中科技大学);School of Computing, National University of Singapore(计算机学院,新加坡国立大学)