arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.10982cs.RO
收藏

Ψ-Map:全景表面集成映射实现真实到仿真转移

Ψ-Map: Panoptic Surface Integrated Mapping Enables Real2Sim Transfer

Department of Control Science and Engineering, Zhejiang University(浙江大学控制科学与工程学院)

Xuan Yu, Yuxuan Xie, Changjian Jiang, Shichao Zhai, Rong Xiong, Yu Zhang, Yue Wang

AI总结:

Ψ-Map通过整合几何强化、端到端学习和高效渲染,实现大规模场景中高精度全景重建,兼顾几何精度与实时推理,满足机器人仿真需求。

摘要:

开放词汇全景重建对高级机器人感知和仿真至关重要。然而,基于3D高斯点划法(3DGS)的方法往往难以在大规模场景中同时实现几何精度、一致的全景理解和实时推理频率。本文提出一个综合框架,整合几何强化、端到端全景学习和高效渲染。首先,为确保大规模环境的物理真实性,利用LiDAR数据构建平面约束的多模态高斯混合模型(GMMs),并采用2D高斯surfels作为地图表示,实现高精度表面对齐和连续几何监督。在此基础上,为克服传统多阶段全景分割管道中的误差累积和繁琐的跨帧关联问题,设计了查询引导的端到端学习架构。通过在视锥体内部利用局部交叉注意力机制,系统将2D掩码特征直接提升到3D空间,实现全局一致的全景理解。最后,为解决高维语义特征导致的计算瓶颈,引入精确瓷砖交集和Top-K硬选择策略以优化渲染流程。实验结果表明,我们的系统在大规模场景中实现了优越的几何和全景重建质量,同时保持推理速度超过40FPS,满足机器人控制回路的实时要求。

英文摘要:

Open-vocabulary panoptic reconstruction is essential for advanced robotics perception and simulation. However, existing methods based on 3D Gaussian Splatting (3DGS) often struggle to simultaneously achieve geometric accuracy, coherent panoptic understanding, and real-time inference frequency in large-scale scenes. In this paper, we propose a comprehensive framework that integrates geometric reinforcement, end-to-end panoptic learning, and efficient rendering. First, to ensure physical realism in large-scale environments, we leverage LiDAR data to construct plane-constrained multimodal Gaussian Mixture Models (GMMs) and employ 2D Gaussian surfels as the map representation, enabling high-precision surface alignment and continuous geometric supervision. Building upon this, to overcome the error accumulation and cumbersome cross-frame association inherent in traditional multi-stage panoptic segmentation pipelines, we design a query-guided end-to-end learning architecture. By utilizing a local cross-attention mechanism within the view frustum, the system lifts 2D mask features directly into 3D space, achieving globally consistent panoptic understanding. Finally, addressing the computational bottlenecks caused by high-dimensional semantic features, we introduce Precise Tile Intersection and a Top-K Hard Selection strategy to optimize the rendering pipeline. Experimental results demonstrate that our system achieves superior geometric and panoptic reconstruction quality in large-scale scenes while maintaining an inference rate exceeding 40 FPS, meeting the real-time requirements of robotic control loops.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑