ObjectGS:基于高斯泼溅的对象感知场景重建与场景理解
ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting
AI总结:
本文提出ObjectGS框架,通过将对象建模为共享ID的局部锚点并结合动态锚点管理与分类损失,统一了3D高斯泼溅的场景重建与语义理解,在开放词汇和全景分割任务上取得领先性能并支持场景编辑等应用。
摘要:
3D高斯泼溅(3D Gaussian Splatting)以其高保真重建和实时新视角合成而闻名,但其缺乏语义理解限制了对象级感知能力。在本工作中,我们提出ObjectGS,一个将3D场景重建与语义理解统一起来的对象感知框架。ObjectGS不将场景视为统一整体,而是将单个对象建模为局部锚点,这些锚点生成神经高斯并共享对象ID,从而实现精确的对象级重建。在训练过程中,我们动态地增长或修剪这些锚点并优化其特征,同时采用带有分类损失的单热ID编码施加明确的语义约束。我们通过大量实验表明,ObjectGS不仅在开放词汇和全景分割任务上优于最先进的方法,还能与网格提取和场景编辑等应用无缝集成。项目页面:https://ruijiezhu94.github.io/ObjectGS_page
英文摘要:
3D Gaussian Splatting is renowned for its high-fidelity reconstructions and real-time novel view synthesis, yet its lack of semantic understanding limits object-level perception. In this work, we propose ObjectGS, an object-aware framework that unifies 3D scene reconstruction with semantic understanding. Instead of treating the scene as a unified whole, ObjectGS models individual objects as local anchors that generate neural Gaussians and share object IDs, enabling precise object-level reconstruction. During training, we dynamically grow or prune these anchors and optimize their features, while a one-hot ID encoding with a classification loss enforces clear semantic constraints. We show through extensive experiments that ObjectGS not only outperforms state-of-the-art methods on open-vocabulary and panoptic segmentation tasks, but also integrates seamlessly with applications like mesh extraction and scene editing. Project page: https://ruijiezhu94.github.io/ObjectGS_page
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。