FreeOcc: 无需训练的具身开放词汇占用预测
FreeOcc: Training-Free Embodied Open-Vocabulary Occupancy Prediction
AI总结：
FreeOcc通过四层流程实现无需3D标注的开放词汇占用预测，相比传统方法在EmbodiedOcc-ScanNet上提升IoU和mIoU超过2倍，并引入ReplicaOcc基准测试新环境性能。
摘要：
现有基于学习的占用预测方法依赖大规模3D标注且泛化能力差。本文提出FreeOcc，一种无需训练的具身开放词汇占用预测框架，从单目或RGB-D序列中进行预测。不同于需要体素级监督和真实相机姿态的先前方法，FreeOcc无需3D标注、姿态真实值或任何学习阶段。FreeOcc通过四层流程逐步构建全局一致的占用地图：SLAM主干估计姿态和稀疏几何；几何一致的高斯更新构建密集的3D高斯地图；开放词汇语义从现成的视觉-语言模型关联到高斯原语；概率高斯到占用的投影产生密集体素占用。尽管完全无需训练且姿态无关，FreeOcc在EmbodiedOcc-ScanNet上相比先前自监督方法在IoU和mIoU上提升超过2倍。我们进一步引入ReplicaOcc，一个用于室内开放词汇占用预测的基准，证明FreeOcc能够零样本迁移到新环境，显著优于监督和自监督基线。项目页面：https://the-masses.github.io/freeocc-web/.
英文摘要：
Existing learning-based occupancy prediction methods rely on large-scale 3D annotations and generalize poorly across environments. We present FreeOcc, a training-free framework for open-vocabulary occupancy prediction from monocular or RGB-D sequences. Unlike prior approaches that require voxel-level supervision and ground-truth camera poses, FreeOcc operates without 3D annotations, pose ground truth, or any learning stage. FreeOcc incrementally builds a globally consistent occupancy map via a four-layer pipeline: a SLAM backbone estimates poses and sparse geometry; a geometrically consistent Gaussian update constructs dense 3D Gaussian maps; open-vocabulary semantics from off-the-shelf vision-language models are associated with Gaussian primitives; and a probabilistic Gaussian-to-occupancy projection produces dense voxel occupancy. Despite being entirely training-free and pose-agnostic, FreeOcc achieves over $2\times$ improvements in IoU and mIoU on EmbodiedOcc-ScanNet compared to prior self-supervised methods. We further introduce ReplicaOcc, a benchmark for indoor open-vocabulary occupancy prediction, and show that FreeOcc transfers zero-shot to novel environments, substantially outperforming both supervised and self-supervised baselines. Project page: https://the-masses.github.io/freeocc-web/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
