3D-STMN：基于依赖驱动的超点-文本匹配网络用于端到端三维指称表达分割
3D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentation
AI总结：
针对3D指称表达分割中两阶段方法的不足，提出端到端超点-文本匹配网络（3D-STMN），利用依赖驱动交互增强语义理解，在ScanRefer上mIoU提升11.7点，推理速度提升95.7倍。
摘要：
在三维指称表达分割（3D-RES）任务中，早期方法采用两阶段范式，即先提取分割提案，再将其与指称表达进行匹配。然而，这种传统范式面临显著挑战，最突出的是初始提案质量不佳以及推理速度明显下降。针对这些局限，我们提出了一种创新的端到端超点-文本匹配网络（3D-STMN），该网络融入了依赖驱动的洞察。我们模型的关键之一在于超点-文本匹配（STM）机制。与传统方法需遍历实例提案不同，STM直接将语言指示与对应的超点（即语义相关的点簇）相关联。这一架构决策使我们的模型能够高效利用跨模态语义关系，主要依赖于密集标注的超点-文本对，而非稀疏的实例-文本对。为了增强文本在引导分割过程中的作用，我们进一步引入了依赖驱动交互（DDI）模块，以加深网络对指称表达的语义理解。该模块以依赖树为指引，辨别表达中主要术语与其相关描述词之间的复杂关系，从而提升模型的定位与分割能力。在ScanRefer基准上的全面实验表明，我们的模型不仅创下了新的性能标准，mIoU提升了11.7个百分点，而且推理速度大幅提升，比传统方法快95.7倍。代码和模型可在https://github.com/sosppxo/3D-STMN获取。
英文摘要：
In 3D Referring Expression Segmentation (3D-RES), the earlier approach adopts a two-stage paradigm, extracting segmentation proposals and then matching them with referring expressions. However, this conventional paradigm encounters significant challenges, most notably in terms of the generation of lackluster initial proposals and a pronounced deceleration in inference speed. Recognizing these limitations, we introduce an innovative end-to-end Superpoint-Text Matching Network (3D-STMN) that is enriched by dependency-driven insights. One of the keystones of our model is the Superpoint-Text Matching (STM) mechanism. Unlike traditional methods that navigate through instance proposals, STM directly correlates linguistic indications with their respective superpoints, clusters of semantically related points. This architectural decision empowers our model to efficiently harness cross-modal semantic relationships, primarily leveraging densely annotated superpoint-text pairs, as opposed to the more sparse instance-text pairs. In pursuit of enhancing the role of text in guiding the segmentation process, we further incorporate the Dependency-Driven Interaction (DDI) module to deepen the network's semantic comprehension of referring expressions. Using the dependency trees as a beacon, this module discerns the intricate relationships between primary terms and their associated descriptors in expressions, thereby elevating both the localization and segmentation capacities of our model. Comprehensive experiments on the ScanRefer benchmark reveal that our model not only set new performance standards, registering an mIoU gain of 11.7 points but also achieve a staggering enhancement in inference speed, surpassing traditional methods by 95.7 times. The code and models are available at https://github.com/sosppxo/3D-STMN.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
