基于视觉基础模型的知识蒸馏的室内点云语义分割可行性
Feasibility of Indoor Frame-Wise Lidar Semantic Segmentation via Distillation from Visual Foundation Model
AI总结：
本文探讨了通过知识蒸馏从视觉基础模型中学习的方法，用于室内点云语义分割，展示了在无人工标注的情况下，该方法在室内场景中的有效性。
摘要：
室内点云帧级语义分割是实现更高层次3D场景理解和映射应用的基础步骤。然而，获取帧级真实标签来训练深度学习模型成本高且耗时。为此，视觉基础模型（VFMs）通过分割图像帧解决了这一问题。通过2D到3D的知识蒸馏流程，相同的VFMs可用于训练点云帧分割模型。在自动驾驶场景中，这种蒸馏方法已取得成功，但在室内场景中尚未验证。本文研究了在室内场景中通过将每个点云扫描与处理过的相机图像耦合，以帧级蒸馏方式重复这一成功。评估使用了室内SLAM数据集，其中伪标签用于下游评估。此外，还提供了一个小规模的手动标注点云数据集用于验证，因为没有其他带有语义的室内点云帧数据集。结果表明，蒸馏模型在伪标签评估中达到56%的mIoU，在真实标签评估中达到约36%的mIoU，证明了在无人工标注情况下，跨模态蒸馏在室内点云语义分割中的可行性。
英文摘要：
Frame-wise semantic segmentation of indoor lidar scans is a fundamental step toward higher-level 3D scene understanding and mapping applications. However, acquiring frame-wise ground truth for training deep learning models is costly and time-consuming. This challenge is largely addressed, for imagery, by Visual Foundation Models (VFMs) which segment image frames. The same VFMs may be used to train a lidar scan frame segmentation model via a 2D-to-3D distillation pipeline. The success of such distillation has been shown for autonomous driving scenes, but not yet for indoor scenes. Here, we study the feasibility of repeating this success for indoor scenes, in a frame-wise distillation manner by coupling each lidar scan with a VFM-processed camera image. The evaluation is done using indoor SLAM datasets, where pseudo-labels are used for downstream evaluation. Also, a small manually annotated lidar dataset is provided for validation, as there are no other lidar frame-wise indoor datasets with semantics. Results show that the distilled model achieves up to 56% mIoU under pseudo-label evaluation and around 36% mIoU with real-label, demonstrating the feasibility of cross-modal distillation for indoor lidar semantic segmentation without manual annotations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
