SeMLaPS：基于潜在先验网络与准平面分割的实时语义建图
SeMLaPS: Real-time Semantic Mapping with Latent Prior Networks and Quasi-Planar Segmentation
AI总结：
提出SeMLaPS方法，结合2D/3D网络与准平面分割，实现RGB-D序列的实时语义建图，在室内数据集上达到最先进质量并支持跨传感器泛化。
摘要：
实时语义信息的可用性极大地提升了SLAM系统的核心几何功能，从而支持众多机器人与AR/VR应用。我们提出了一种从RGB-D序列进行实时语义建图的新方法，该方法结合了基于3D占用建图的SLAM系统上的2D神经网络与3D网络。在分割新帧时，我们基于可微渲染从先前帧进行潜在特征重投影。将先前帧的重投影特征图与当前帧特征融合，相比独立处理图像的基线，显著提高了图像分割质量。对于3D地图处理，我们提出了一种新颖的几何准平面过分割方法，该方法依赖表面法线将可能属于相同语义类别的3D地图元素分组。我们还描述了一种用于轻量级语义地图后处理的新型神经网络设计。我们的系统在基于2D-3D网络的系统中达到了最先进的语义建图质量，并在三个真实室内数据集上与3D卷积网络性能相当，同时实现实时运行。此外，与3D CNN相比，它展示了更好的跨传感器泛化能力，支持不同深度传感器的训练与推理。代码和数据将在项目页面发布：http://jingwenwang95.github.io/SeMLaPS
英文摘要：
The availability of real-time semantics greatly improves the core geometric functionality of SLAM systems, enabling numerous robotic and AR/VR applications. We present a new methodology for real-time semantic mapping from RGB-D sequences that combines a 2D neural network and a 3D network based on a SLAM system with 3D occupancy mapping. When segmenting a new frame we perform latent feature re-projection from previous frames based on differentiable rendering. Fusing re-projected feature maps from previous frames with current-frame features greatly improves image segmentation quality, compared to a baseline that processes images independently. For 3D map processing, we propose a novel geometric quasi-planar over-segmentation method that groups 3D map elements likely to belong to the same semantic classes, relying on surface normals. We also describe a novel neural network design for lightweight semantic map post-processing. Our system achieves state-of-the-art semantic mapping quality within 2D-3D networks-based systems and matches the performance of 3D convolutional networks on three real indoor datasets, while working in real-time. Moreover, it shows better cross-sensor generalization abilities compared to 3D CNNs, enabling training and inference with different depth sensors. Code and data will be released on project page: http://jingwenwang95.github.io/SeMLaPS
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
