arXivDaily arXiv每日学术速递 周一至周五更新
​通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。
arXiv 2207.01821cs.CV

通过指称文本短语实现可解释且细粒度的3D定位

Toward Explainable and Fine-Grained 3D Grounding through Referring Textual Phrases

CUHK-Shenzhen（香港中文大学（深圳））；Shanghai Jiao Tong University（上海交通大学）

Zhihao Yuan, Xu Yan, Zhuo Li, Xuhao Li, Yao Guo, Shuguang Cui, Zhen Li

备注：
New dataset for 3D visual grounding is available at https://yanx27.github.io/phraserefer/
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出3D短语感知定位（3DPAG）任务，通过构建约227K短语级标注数据集，并设计短语-物体对齐优化与短语特定预训练方法，在Nr3D、Sr3D和ScanRefer上显著提升了3D视觉定位性能。

摘要：

近期3D场景理解的进展探索了视觉定位（3DVG），即通过语言描述来定位目标物体。然而，现有方法仅考虑整个句子与目标物体之间的依赖关系，忽略了上下文与非目标物体之间的细粒度关系。在本文中，我们将3DVG扩展为一个更细粒度和可解释的任务，称为3D短语感知定位（3DPAG）。3DPAG任务旨在通过显式识别所有与短语相关的物体，然后根据上下文短语进行推理，从而在3D场景中定位目标物体。为解决这一问题，我们使用自主开发的平台，从广泛使用的3DVG数据集（即Nr3D、Sr3D和ScanRefer）的88K条句子中，人工标注了约227K个短语级标注。通过利用我们的数据集，可以将先前的3DVG方法扩展到细粒度的短语感知场景。这一目标通过所提出的新颖短语-物体对齐优化和短语特定预训练来实现，同时也提升了传统3DVG的性能。大量实验结果证实了显著的改进，即先前最先进的方法在Nr3D、Sr3D和ScanRefer上的总体准确率分别提升了3.9%、3.5%和4.6%。

英文摘要：

Recent progress in 3D scene understanding has explored visual grounding (3DVG) to localize a target object through a language description. However, existing methods only consider the dependency between the entire sentence and the target object, ignoring fine-grained relationships between contexts and non-target ones. In this paper, we extend 3DVG to a more fine-grained and interpretable task, called 3D Phrase Aware Grounding (3DPAG). The 3DPAG task aims to localize the target objects in a 3D scene by explicitly identifying all phrase-related objects and then conducting the reasoning according to contextual phrases. To tackle this problem, we manually labeled about 227K phrase-level annotations using a self-developed platform, from 88K sentences of widely used 3DVG datasets, i.e., Nr3D, Sr3D and ScanRefer. By tapping on our datasets, we can extend previous 3DVG methods to the fine-grained phrase-aware scenario. It is achieved through the proposed novel phrase-object alignment optimization and phrase-specific pre-training, boosting conventional 3DVG performance as well. Extensive results confirm significant improvements, i.e., previous state-of-the-art method achieves 3.9%, 3.5% and 4.6% overall accuracy gains on Nr3D, Sr3D and ScanRefer respectively.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑