arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.14232cs.CV
收藏

S2GS:流式语义高斯点云用于在线场景理解和重建

S2GS: Streaming Semantic Gaussian Splatting for Online Scene Understanding and Reconstruction

Renhe Zhang, Yuyang Tan, Jingyu Gong, Zhizhong Zhang, Lizhuang Ma, Yuan Xie, Xin Tan

AI总结:

S2GS提出一种严格因果的增量3D高斯语义场框架,通过分离几何与语义的双主干设计,实现在线场景重建与理解,显著提升长视界可扩展性。

摘要:

现有针对长图像流的联合场景理解和重建的离线前馈方法通常反复对不断增长的过去观测集进行全局计算,导致运行时间和GPU内存随序列长度快速增加,限制了可扩展性。我们提出流式语义高斯点云(S2GS),一种严格因果、增量的3D高斯语义场框架:它不利用未来帧,且无需重新处理历史帧即可持续更新场景几何、外观和实例级语义,从而实现可扩展的在线联合重建与理解。S2GS采用几何-语义解耦的双主干设计:几何分支进行因果建模以驱动增量高斯更新,而语义分支利用2D基础视觉模型和查询驱动的解码器预测分割掩码和身份嵌入,进一步通过查询级对比对齐和轻量级在线关联与实例内存进行稳定。实验表明,S2GS在联合重建与理解基准上与强离线基线匹配或优于,同时显著提升长视界可扩展性:它能处理1000+帧,且运行时间和GPU内存增长远低于离线全局处理基线,在相同设置下,离线全局处理基线通常在80帧左右就因内存不足而无法继续运行。

英文摘要:

Existing offline feed-forward methods for joint scene understanding and reconstruction on long image streams often repeatedly perform global computation over an ever-growing set of past observations, causing runtime and GPU memory to increase rapidly with sequence length and limiting scalability. We propose Streaming Semantic Gaussian Splatting (S2GS), a strictly causal, incremental 3D Gaussian semantic field framework: it does not leverage future frames and continuously updates scene geometry, appearance, and instance-level semantics without reprocessing historical frames, enabling scalable online joint reconstruction and understanding. S2GS adopts a geometry-semantic decoupled dual-backbone design: the geometry branch performs causal modeling to drive incremental Gaussian updates, while the semantic branch leverages a 2D foundation vision model and a query-driven decoder to predict segmentation masks and identity embeddings, further stabilized by query-level contrastive alignment and lightweight online association with an instance memory. Experiments show that S2GS matches or outperforms strong offline baselines on joint reconstruction-and-understanding benchmarks, while significantly improving long-horizon scalability: it processes 1,000+ frames with much slower growth in runtime and GPU memory, whereas offline global-processing baselines typically run out of memory at around 80 frames under the same setting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑