3D-GRES：广义三维指代表达分割
3D-GRES: Generalized 3D Referring Expression Segmentation
AI总结：
本文提出广义三维指代表达分割任务3D-GRES，通过多查询解耦交互网络MDIN将多目标分割分解为单目标分割，并构建Multi3DRes数据集，显著提升多目标三维场景理解性能。
摘要：
三维指代表达分割（3D-RES）致力于根据自然语言描述在三维空间中分割特定实例。然而，当前方法仅限于分割单一目标，限制了该任务的通用性。为克服这一局限，我们提出了广义三维指代表达分割（3D-GRES），将能力扩展到基于自然语言指令分割任意数量的实例。针对这一更广泛的任务，我们提出了多查询解耦交互网络（MDIN），旨在将多目标分割任务分解为更简单的单目标分割。MDIN包含两个基本组件：文本驱动的稀疏查询（TSQ）和多目标解耦优化（MDO）。TSQ生成分布在关键目标上的稀疏点云特征作为查询的初始化。同时，MDO负责在多目标场景中将每个目标分配给不同的查询，同时保持其语义一致性。为适应这一新任务，我们构建了一个新数据集，即Multi3DRes。我们在该数据集上的综合评估表明，相较于现有模型取得了显著提升，从而为复杂的多目标三维场景理解开辟了新路径。基准和代码可在 https://github.com/sosppxo/MDIN 获取。
英文摘要：
3D Referring Expression Segmentation (3D-RES) is dedicated to segmenting a specific instance within a 3D space based on a natural language description. However, current approaches are limited to segmenting a single target, restricting the versatility of the task. To overcome this limitation, we introduce Generalized 3D Referring Expression Segmentation (3D-GRES), which extends the capability to segment any number of instances based on natural language instructions. In addressing this broader task, we propose the Multi-Query Decoupled Interaction Network (MDIN), designed to break down multi-object segmentation tasks into simpler, individual segmentations. MDIN comprises two fundamental components: Text-driven Sparse Queries (TSQ) and Multi-object Decoupling Optimization (MDO). TSQ generates sparse point cloud features distributed over key targets as the initialization for queries. Meanwhile, MDO is tasked with assigning each target in multi-object scenarios to different queries while maintaining their semantic consistency. To adapt to this new task, we build a new dataset, namely Multi3DRes. Our comprehensive evaluations on this dataset demonstrate substantial enhancements over existing models, thus charting a new path for intricate multi-object 3D scene comprehension. The benchmark and code are available at https://github.com/sosppxo/MDIN.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
