四足机器人全景多模态语义占用预测
Panoramic Multimodal Semantic Occupancy Prediction for Quadruped Robots
AI总结：
本文提出PanoMMOcc数据集和VoxelHound框架，通过垂直抖动补偿和多模态信息融合提升四足机器人全景多模态3D感知性能。
摘要：
全景影像为四足机器人提供全景视觉覆盖。然而现有占用预测方法主要用于轮式自动驾驶，依赖RGB信息，限制了复杂环境下的鲁棒性。为弥合这一差距，本文提出了首个适用于四足机器人的全景多模态占用数据集PanoMMOcc，包含四种传感模态；提出全景多模态占用感知框架VoxelHound，专为腿式移动和球形成像设计。具体包括垂直抖动补偿模块和多模态信息提示融合模块，以提升体积占用预测。本文建立了基于PanoMMOcc的基准，并提供详细数据分析以系统评估感知方法在挑战性具身场景下的性能。大量实验表明，VoxelHound在PanoMMOcc上达到mIoU的最新水平（+4.16%）。数据集和代码将在https://github.com/SXDR/PanoMMOcc公开发布，同时附带校准工具https://github.com/losehu/CameraLiDAR-Calib。
英文摘要：
Panoramic imagery provides holistic 360° visual coverage for environmental perception in quadruped robots. However, existing occupancy prediction methods are primarily designed for wheeled autonomous driving and rely heavily on RGB cues, which limits their robustness in complex, dynamically changing environments. To bridge this gap, we introduce PanoMMOcc, the first real-world panoramic multimodal occupancy dataset for quadruped robots, comprising four sensing modalities collected across diverse scenes. We further propose VoxelHound, a panoramic multimodal occupancy perception framework tailored to legged locomotion and spherical imaging. VoxelHound incorporates a Vertical Jitter Compensation (VJC) module to mitigate severe viewpoint perturbations caused by body pitch and roll during locomotion, enabling more consistent spatial reasoning, and a Multimodal Information Prompt Fusion (MIPF) module to effectively integrate panoramic visual cues with auxiliary modalities for enhanced volumetric occupancy prediction. We also establish a comprehensive benchmark on PanoMMOcc and provide detailed dataset analyses to enable systematic evaluation in challenging embodied perception scenarios. Extensive experiments demonstrate that VoxelHound achieves state-of-the-art performance on PanoMMOcc, with a +4.16 gain in mIoU. The dataset and code will be publicly released to facilitate future research on panoramic multimodal 3D perception for embodied robotic systems at https://github.com/SXDR/PanoMMOcc.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
