RAZER：基于时空聚合的鲁棒加速零样本3D开放词汇全景重建
RAZER: Robust Accelerated Zero-Shot 3D Open-Vocabulary Panoptic Reconstruction with Spatio-Temporal Aggregation
AI总结：
针对现有3D语义建图系统缺乏在线开放词汇构建能力的问题，本文提出无需训练的RAZER框架，通过分层物体关联引导的在线实例级语义嵌入融合，整合GPU加速几何重建与开放词汇视觉语言模型，实现鲁棒实时的3D场景理解，支持多种下游任务。
摘要：
对复杂3D环境进行建图与理解是自主系统感知物理世界并与之交互的基础，既需要精确的几何重建，也需要丰富的语义理解。现有3D语义建图系统虽擅长重建和识别预定义的物体实例，但缺乏在线运行时高效构建开放词汇语义地图的灵活性。尽管近期的视觉语言模型已实现2D图像中的开放词汇物体识别，但它们尚未打通与3D空间理解之间的壁垒。核心挑战在于开发一套无需训练的统一系统，既能同时构建精确的3D地图，又能保持语义一致性，还支持实时自然语言交互。本文提出了一个零样本框架，通过在线实例级语义嵌入融合，在带空间索引的分层物体关联引导下，将GPU加速的几何重建与开放词汇视觉语言模型无缝整合。这套无需训练的系统通过增量处理和统一的几何-语义更新实现了优异性能，同时能鲁棒地处理2D分割不一致问题。所提出的通用3D场景理解框架可用于零样本3D实例检索、分割和物体检测等多种任务，能够推理从未见过的物体并解析自然语言查询。项目页面见https://razer-3d.github.io。
英文摘要：
Mapping and understanding complex 3D environments is fundamental to how autonomous systems perceive and interact with the physical world, requiring both precise geometric reconstruction and rich semantic comprehension. While existing 3D semantic mapping systems excel at reconstructing and identifying predefined object instances, they lack the flexibility to efficiently build semantic maps with open-vocabulary during online operation. Although recent vision-language models have enabled open-vocabulary object recognition in 2D images, they haven't yet bridged the gap to 3D spatial understanding. The critical challenge lies in developing a training-free unified system that can simultaneously construct accurate 3D maps while maintaining semantic consistency and supporting natural language interactions in real time. In this paper, we develop a zero-shot framework that seamlessly integrates GPU-accelerated geometric reconstruction with open-vocabulary vision-language models through online instance-level semantic embedding fusion, guided by hierarchical object association with spatial indexing. Our training-free system achieves superior performance through incremental processing and unified geometric-semantic updates, while robustly handling 2D segmentation inconsistencies. The proposed general-purpose 3D scene understanding framework can be used for various tasks including zero-shot 3D instance retrieval, segmentation, and object detection to reason about previously unseen objects and interpret natural language queries. The project page is available at https://razer-3d.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
