AugRefer：通过跨模态增强与基于空间关系的指代推进3D视觉定位
AugRefer: Advancing 3D Visual Grounding via Cross-Modal Augmentation and Spatial Relation-based Referring
AI总结：
针对3D视觉定位训练文本-3D对不足、空间关系利用不充分的问题，提出AugRefer方法，通过跨模态增强扩充训练数据，并设计语言-空间自适应解码器，经多基准实验验证有效提升了3D视觉定位性能。
摘要：
3D视觉定位（3DVG）旨在将自然语言描述与3D场景中的目标对象关联起来，是一项重要且具有挑战性的任务。尽管该领域近期取得了进展，但现有方法普遍存在一个不足：可用于训练的文本-3D对数量有限且多样性不足。此外，它们未能有效利用不同的上下文线索（例如3D视觉空间内丰富的空间关系）来完成定位。为解决这些局限，我们提出了AugRefer，一种推进3D视觉定位的新方法。AugRefer引入了跨模态增强，旨在通过将对象放置到3D场景中，并利用基础模型生成准确且语义丰富的描述，来大量生成多样化的文本-3D对。值得注意的是，生成的配对可被任何现有3DVG方法用于丰富其训练数据。此外，AugRefer提出了一种语言-空间自适应解码器，能够根据语言描述和各种3D空间关系，有效调整潜在的指代对象。在三个基准数据集上开展的大量实验清晰验证了AugRefer的有效性。
英文摘要：
3D visual grounding (3DVG), which aims to correlate a natural language description with the target object within a 3D scene, is a significant yet challenging task. Despite recent advancements in this domain, existing approaches commonly encounter a shortage: a limited amount and diversity of text3D pairs available for training. Moreover, they fall short in effectively leveraging different contextual clues (e.g., rich spatial relations within the 3D visual space) for grounding. To address these limitations, we propose AugRefer, a novel approach for advancing 3D visual grounding. AugRefer introduces cross-modal augmentation designed to extensively generate diverse text-3D pairs by placing objects into 3D scenes and creating accurate and semantically rich descriptions using foundation models. Notably, the resulting pairs can be utilized by any existing 3DVG methods for enriching their training data. Additionally, AugRefer presents a language-spatial adaptive decoder that effectively adapts the potential referring objects based on the language description and various 3D spatial relations. Extensive experiments on three benchmark datasets clearly validate the effectiveness of AugRefer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
