arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.10951cs.RO
收藏

Fast-SegSim: 仿真中机器人快速开放词汇分割

Fast-SegSim: Real-Time Open-Vocabulary Segmentation for Robotics in Simulation

Zhejiang University(浙江大学)

Xuan Yu, Yuxuan Xie, Shichao Zhai, Shuhao Ye, Rong Xiong, Yue Wang

AI总结:

本文提出Fast-SegSim,基于2D高斯散射构建端到端框架,实现高保真且3D一致的开放词汇分割重建。通过优化渲染管线和Top-K硬选择策略,提升渲染速度至40FPS以上,用于仿真平台传感器输入和下游感知任务的多视图地面真实标签生成。

摘要:

开放词汇全景重建对先进机器人和仿真至关重要。然而,现有3D重建方法如NeRF或高斯点划变种难以满足机器人控制回路所需的实时推理频率。现有方法在处理高维特征时存在显著延迟。我们提出Fast-SegSim,一种新颖、简单且端到端的框架,基于2D高斯散射,实现实时、高保真且3D一致的开放词汇分割重建。我们的核心贡献是高度优化的渲染管线,专门解决高通道分割特征积累的计算瓶颈。我们引入两种关键优化:精确瓷砖相交以减少光栅化冗余,以及新颖的Top-K硬选择策略。该策略利用2D高斯表示中固有的几何稀疏性,大大简化特征积累并缓解带宽限制,实现渲染速度超过40 FPS。Fast-SegSim在机器人应用中具有关键价值:它既为Gazebo等仿真平台提供高频传感器输入,其3D一致的输出为下游感知任务的微调提供必要的多视图'地面真实'标签。我们通过使用生成的标签微调目标导航模块,成功将导航成功率翻倍。我们的优越渲染速度和实用价值凸显了Fast-SegSim在仿真到现实差距中的潜力。

英文摘要:

Open-vocabulary panoptic reconstruction is crucial for advanced robotics and simulation. However, existing 3D reconstruction methods, such as NeRF or Gaussian Splatting variants, often struggle to achieve the real-time inference frequency required by robotic control loops. Existing methods incur prohibitive latency when processing the high-dimensional features required for robust open-vocabulary segmentation. We propose Fast-SegSim, a novel, simple, and end-to-end framework built upon 2D Gaussian Splatting, designed to realize real-time, high-fidelity, and 3D-consistent open-vocabulary segmentation reconstruction. Our core contribution is a highly optimized rendering pipeline that specifically addresses the computational bottleneck of high-channel segmentation feature accumulation. We introduce two key optimizations: Precise Tile Intersection to reduce rasterization redundancy, and a novel Top-K Hard Selection strategy. This strategy leverages the geometric sparsity inherent in the 2D Gaussian representation to greatly simplify feature accumulation and alleviate bandwidth limitations, achieving render rates exceeding 40 FPS. Fast-SegSim provides critical value in robotic applications: it serves both as a high-frequency sensor input for simulation platforms like Gazebo, and its 3D-consistent outputs provide essential multi-view 'ground truth' labels for fine-tuning downstream perception tasks. We demonstrate this utility by using the generated labels to fine-tune the perception module in object goal navigation, successfully doubling the navigation success rate. Our superior rendering speed and practical utility underscore Fast-SegSim's potential to bridge the sim-to-real gap.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑