基于视觉-语言模型的点云地图定位
VLM-Loc: Localization in Point Cloud Maps via Vision-Language Models
AI总结：
VLM-Loc通过视觉-语言模型的空间推理能力，实现了更准确的文本到点云定位，提升了复杂环境中的定位鲁棒性。
摘要：
文本到点云（T2P）定位旨在从自然语言描述中推断出3D点云地图中的精确空间位置，反映人类如何通过语言感知和沟通空间布局。然而，现有方法大多依赖于浅层的文本-点云对应关系，缺乏有效的空间推理能力，限制了其在复杂环境中的准确性。为了解决这一限制，我们提出了VLM-Loc框架，该框架利用大型视觉-语言模型（VLMs）的空间推理能力进行T2P定位。具体来说，我们将点云转换为鸟瞰图（BEV）图像和场景图，这些图像和图共同编码了几何和语义上下文，为VLM提供结构化的输入，以学习跨模态表示，连接语言和空间语义。在这些表示之上，我们引入了一种部分节点分配机制，该机制明确地将文本提示与场景图节点关联起来，使空间推理变得可解释，从而实现准确的定位。为了在多样化的场景中进行系统评估，我们提出了CityLoc基准，该基准由多源点云构建，用于细粒度的T2P定位。在CityLoc上的实验表明，VLM-Loc在准确性和鲁棒性方面优于最先进的方法。我们的代码、模型和数据集可在https://github.com/MCG-NKU/nku-3d-vision仓库中找到。
英文摘要：
Text-to-point-cloud (T2P) localization aims to infer precise spatial positions within 3D point cloud maps from natural language descriptions, reflecting how humans perceive and communicate spatial layouts through language. However, existing methods largely rely on shallow text-point cloud correspondence without effective spatial reasoning, limiting their accuracy in complex environments. To address this limitation, we propose VLM-Loc, a framework that leverages the spatial reasoning capability of large vision-language models (VLMs) for T2P localization. Specifically, we transform point clouds into bird's-eye-view (BEV) images and scene graphs that jointly encode geometric and semantic context, providing structured inputs for the VLM to learn cross-modal representations bridging linguistic and spatial semantics. On top of these representations, we introduce a partial node assignment mechanism that explicitly associates textual cues with scene graph nodes, enabling interpretable spatial reasoning for accurate localization. To facilitate systematic evaluation across diverse scenes, we present CityLoc, a benchmark built from multi-source point clouds for fine-grained T2P localization. Experiments on CityLoc demonstrate VLM-Loc achieves superior accuracy and robustness compared to state-of-the-art methods. Our code, model, and dataset are available at \href{https://github.com/MCG-NKU/nku-3d-vision}{repository}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
