少样本增量式动态室内环境3D物体检测
Few-Shot Incremental 3D Object Detection in Dynamic Indoor Environments
AI总结：
本文提出FI3Det框架，利用视觉语言模型学习未见类别知识，通过VLM引导模块和多模态原型模块实现高效3D感知，在ScanNet V2和SUN RGB-D数据集上取得显著提升。
摘要：
增量式3D感知是实现动态室内环境中具身智能的关键步骤。然而，现有方法依赖大量新类标注以获得满意性能。为解决此问题，我们提出FI3Det框架，通过利用视觉语言模型（VLMs）学习未见类别的知识，仅需少量新样本即可实现高效3D感知。FI3Det在基础阶段引入VLM引导的未知物体学习模块，利用VLM挖掘未知物体并提取综合表示，包括2D语义特征和类无关的3D包围盒。为减轻这些表示中的噪声，设计了加权机制以重新加权点和盒级特征的贡献基于其空间位置和特征一致性。此外，FI3Det提出一个门控多模态原型印记模块，通过对齐的2D语义和3D几何特征构建类别原型以计算分类分数，然后通过多模态门控机制融合以实现新物体检测。作为首个少样本增量式3D物体检测框架，我们在两个数据集上建立了批量和序列评估设置，FI3Det在基线方法上实现了显著且一致的提升。代码可在https://github.com/zyrant/FI3Det获取。
英文摘要：
Incremental 3D object perception is a critical step toward embodied intelligence in dynamic indoor environments. However, existing incremental 3D detection methods rely on extensive annotations of novel classes for satisfactory performance. To address this limitation, we propose FI3Det, a Few-shot Incremental 3D Detection framework that enables efficient 3D perception with only a few novel samples by leveraging vision-language models (VLMs) to learn knowledge of unseen categories. FI3Det introduces a VLM-guided unknown object learning module in the base stage to enhance perception of unseen categories. Specifically, it employs VLMs to mine unknown objects and extract comprehensive representations, including 2D semantic features and class-agnostic 3D bounding boxes. To mitigate noise in these representations, a weighting mechanism is further designed to re-weight the contributions of point- and box-level features based on their spatial locations and feature consistency within each box. Moreover, FI3Det proposes a gated multimodal prototype imprinting module, where category prototypes are constructed from aligned 2D semantic and 3D geometric features to compute classification scores, which are then fused via a multimodal gating mechanism for novel object detection. As the first framework for few-shot incremental 3D object detection, we establish both batch and sequential evaluation settings on two datasets, ScanNet V2 and SUN RGB-D, where FI3Det achieves strong and consistent improvements over baseline methods. Code is available at https://github.com/zyrant/FI3Det.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
