提升密集三维视觉定位中视觉-语言融合的四种方法
Four Ways to Improve Verbo-visual Fusion for Dense 3D Visual Grounding
AI总结：
针对密集三维视觉定位，提出ConcreteNet网络，通过四种创新模块提升重复实例的定位性能，在ScanRefer基准上排名第一。
摘要：
三维视觉定位是在三维场景中定位由自然语言描述所指对象的任务。该任务在从自主室内机器人到增强现实/虚拟现实等广泛应用中，近来日益受到关注。解决三维视觉定位的一种常见方法是基于检测的定位，即通过边界框进行定位。然而，对于需要物理交互的现实应用，边界框不足以描述对象的几何形状。因此，我们着手解决密集三维视觉定位问题，即基于指称的三维实例分割。我们提出了一种密集三维定位网络ConcreteNet，其具有四个新颖的独立模块，旨在提升对具有挑战性的重复实例（即具有相同语义类别的干扰物的实例）的定位性能。首先，我们引入了一个自底向上的注意力融合模块，旨在消除实例间关系线索的歧义；其次，我们构建了一个对比训练方案以在潜在空间中诱导分离；然后，我们通过一个学习的全局相机标记来解决视角相关的表述；最后，我们采用多视角集成来提高所指掩码的质量。ConcreteNet在具有挑战性的ScanRefer在线基准上排名第一，并赢得了ICCV第三届三维场景语言研讨会“三维对象定位”挑战赛。我们的代码可在ouenal.github.io/concretenet/获取。
英文摘要：
3D visual grounding is the task of localizing the object in a 3D scene which is referred by a description in natural language. With a wide range of applications ranging from autonomous indoor robotics to AR/VR, the task has recently risen in popularity. A common formulation to tackle 3D visual grounding is grounding-by-detection, where localization is done via bounding boxes. However, for real-life applications that require physical interactions, a bounding box insufficiently describes the geometry of an object. We therefore tackle the problem of dense 3D visual grounding, i.e. referral-based 3D instance segmentation. We propose a dense 3D grounding network ConcreteNet, featuring four novel stand-alone modules that aim to improve grounding performance for challenging repetitive instances, i.e. instances with distractors of the same semantic class. First, we introduce a bottom-up attentive fusion module that aims to disambiguate inter-instance relational cues, next, we construct a contrastive training scheme to induce separation in the latent space, we then resolve view-dependent utterances via a learned global camera token, and finally we employ multi-view ensembling to improve referred mask quality. ConcreteNet ranks 1st on the challenging ScanRefer online benchmark and has won the ICCV 3rd Workshop on Language for 3D Scenes "3D Object Localization" challenge. Our code is available at ouenal.github.io/concretenet/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
