DGSG-Mind：用于长期场景理解与定位的动态3D高斯场景图
DGSG-Mind: Dynamic 3D Gaussian Scene Graphs for Long-Term Scene Understanding and Grounding
AI总结：
提出DGSG-Mind，一种混合实例感知的3D高斯动态场景图系统，通过概率体素网格与显式3D高斯结合实现鲁棒的跨模态实例融合和增量语义映射，并构建层次化场景图与3D高斯思维进行多模态推理，在零样本3D视觉定位、开放词汇语义分割和场景重建中取得领先性能。
摘要：
将开放词汇语义信息集成到动态3D场景表示中对于长期具身场景理解至关重要。然而，现有方法常因跨视角线索不完整而导致脆弱的实例关联，同时处理对象级拓扑变化的能力有限，限制了长期机器人任务执行。此外，当前的3D场景理解方法要么依赖简单的特征匹配而缺乏显式空间推理，要么假设离线真实3D几何。为应对这些挑战，我们提出DGSG-Mind，一种混合实例感知的3D高斯动态场景图系统，配备具身推理智能体。我们的系统将概率体素网格与显式3D高斯耦合，实现鲁棒的跨模态实例融合和增量语义映射。它通过基于高斯的视觉重定位和由几何-语义一致性引导的局部掩码细化来处理动态变化。基于实例高斯图，DGSG-Mind进一步构建层次化场景图，并开发3D高斯思维，集成结构关系、空间-语义信息和视觉标注的RoI高斯渲染以进行多模态推理。大量实验表明，DGSG-Mind在基于自重建地图的方法中实现了最佳的零样本3D视觉定位性能，同时在3D开放词汇语义分割和场景重建中也表现出强劲性能。我们进一步将DGSG-Mind部署到真实世界机器人上，展示其目标导向推理和动态更新能力。DGSG-Mind的项目页面位于https://icr-lab.github.io/DGSG-Mind。
英文摘要：
Integrating open-vocabulary semantic information into dynamic 3D scene representations is essential for long-term embodied scene understanding. However, existing methods often suffer from fragile instance association due to incomplete cross-view cues, while their limited ability to handle object-level topological changes restricts long-term robotic task execution. Moreover, current 3D scene understanding methods either rely on simple feature matching without explicit spatial reasoning or assume offline ground-truth 3D geometry. To address these challenges, we present DGSG-Mind, a hybrid instance-aware 3D Gaussian dynamic scene graph system with an embodied reasoning agent. Our system couples a probabilistic voxel grid with explicit 3D Gaussians to enable robust cross-modal instance fusion and incremental semantic mapping. It handles dynamic changes through Gaussian-based visual relocalization and localized masked refinement guided by geometric-semantic consistency. Built on the instance Gaussian map, DGSG-Mind further constructs a hierarchical scene graph and develops the 3D Gaussian Mind, which integrates structural relations, spatial-semantic information, and visually annotated RoI Gaussian renderings for multimodal reasoning. Extensive experiments show that DGSG-Mind achieves the best zero-shot 3DVG performance among methods operating on self-reconstructed maps, while also delivering strong performance in 3D open-vocabulary semantic segmentation and scene reconstruction. We further deploy DGSG-Mind on real-world robots to demonstrate its target-oriented reasoning and dynamic update capabilities. The project page of DGSG-Mind is available at https://icr-lab.github.io/DGSG-Mind
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
