arXiv 2604.262612026-04-29cs.CV

多一致的2D-3D映射用于鲁棒的零样本3D视觉定位

Multiple Consistent 2D-3D Mappings for Robust Zero-Shot 3D Visual Grounding

Hangzhou Dianzi University（杭州电子科技大学）；Zhejiang University（浙江大学）；Alibaba Group（阿里巴巴集团）；Institute for Clarity in Documentation（文档清晰研究所）；Inria Paris-Rocquencourt（Inria巴黎-鲁克蒙特研究所）；Rajiv Gandhi University（拉贾·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒实验室）

Yufei Yin, Jie Zheng, Qianke Meng, Zhou Yu, Minghao Chen, Jiajun Ding, Min Tan, Yuling Xi, Zhiwen Chen, Chengfei Lv

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出MCM-VG框架，通过建立多一致的2D-3D映射实现鲁棒的零样本3D视觉定位，解决开放词汇3D提案质量差的问题，提升目标定位和推理的准确性。

摘要：

本文提出MCM-VG框架，通过建立多一致的2D-3D映射实现鲁棒的零样本3D视觉定位，解决开放词汇3D提案质量差的问题，提升目标定位和推理的准确性。

英文摘要：

Zero-shot 3D Visual Grounding (3DVG) is a critical capability for open-world embodied AI. However, existing methods are fundamentally bottlenecked by the poor quality of open-vocabulary 3D proposals, suffering from inaccurate categories and imprecise geometries, as well as the spatial redundancy of exhaustive multi-view reasoning. To address these challenges, we propose MCM-VG, a novel framework that achieves robust zero-shot 3DVG by explicitly establishing Multiple Consistent 2D-3D Mappings. Instead of passively relying on noisy 3D segments, MCM-VG enforces 2D-3D consistency across three fundamental dimensions to achieve precise target localization and reliable reasoning. First, a Semantic Alignment module corrects category mismatches via LLM-driven query parsing and coarse-to-fine 2D-3D matching. Second, an Instance Rectification module leverages VLM-guided 2D segmentations to reconstruct missing targets, back-projecting these reliable visual priors to establish accurate 3D geometries. Finally, to eliminate spatial redundancy, a Viewpoint Distillation module clusters 3D camera directions to extract optimal frames. By pairing these optimal RGB frames with Bird's Eye View maps into concise visual prompt sets, we formulate the final target disambiguation as a multiple-choice reasoning task for Vision-Language Models. Extensive evaluations on ScanRefer and Nr3D benchmarks demonstrate that MCM-VG sets a new state-of-the-art for zero-shot 3D visual grounding. Remarkably, it achieves 62.0\% and 53.6\% in Acc@0.25 and Acc@0.5 on ScanRefer, outperforming previous baselines by substantial margins of 6.4\% and 4.0\%.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。