Multi3DRefer：将文本描述定位到多个3D对象
Multi3DRefer: Grounding Text Description to Multiple 3D Objects
AI总结：
提出Multi3DRefer任务及数据集，解决真实3D场景中灵活数量对象的自然语言定位问题，并开发基于CLIP和对比学习的基线方法在ScanRefer基准上达到SOTA。
摘要：
我们引入了使用自然语言描述在真实世界3D场景中定位灵活数量对象的任务。现有的3D视觉定位任务侧重于在给定文本描述时定位唯一对象。然而，这种严格的设定是不自然的，因为定位潜在多个对象是真实场景和机器人任务（例如视觉导航和对象重排）中的常见需求。为了解决这一设定，我们提出了Multi3DRefer，对ScanRefer数据集和任务进行了泛化。我们的数据集包含针对11609个对象的61926条描述，其中每条描述引用零个、单个或多个目标对象。我们还引入了新的评估指标，并对先前工作的方法进行了基准测试，以促进对多模态3D场景理解的进一步研究。此外，我们开发了一个更好的基线方法，通过对比学习在线渲染对象提议，利用来自CLIP的2D特征，该方法在ScanRefer基准上超越了现有技术水平。
英文摘要：
We introduce the task of localizing a flexible number of objects in real-world 3D scenes using natural language descriptions. Existing 3D visual grounding tasks focus on localizing a unique object given a text description. However, such a strict setting is unnatural as localizing potentially multiple objects is a common need in real-world scenarios and robotic tasks (e.g., visual navigation and object rearrangement). To address this setting we propose Multi3DRefer, generalizing the ScanRefer dataset and task. Our dataset contains 61926 descriptions of 11609 objects, where zero, single or multiple target objects are referenced by each description. We also introduce a new evaluation metric and benchmark methods from prior work to enable further investigation of multi-modal 3D scene understanding. Furthermore, we develop a better baseline leveraging 2D features from CLIP by rendering object proposals online with contrastive learning, which outperforms the state of the art on the ScanRefer benchmark.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
