CoT3DRef：基于思维链的数据高效型3D视觉定位
CoT3DRef: Chain-of-Thoughts Data-Efficient 3D Visual Grounding
AI总结：
针对现有3D视觉定位方法可解释性差、复杂场景易失效的问题，提出基于思维链的CoT3DRef框架，将任务拆解为可解释的中间步骤，在多个基准上实现性能提升，且数据效率极高。
摘要：
3D视觉定位指的是根据自然语言描述在3D场景中定位目标物体的能力。现有大多数方法通过指代头直接定位被指代物体，在复杂场景下容易失效，且无法说明网络得出最终决策的过程与原因。本文针对“能否设计一套可模拟人类感知系统的可解释3D视觉定位框架”这一问题展开研究。为此，我们将3D视觉定位问题建模为序列到序列（Seq2Seq）任务，先预测一系列锚点链，再输出最终目标。可解释性不仅能提升整体性能，还能帮助我们识别失败案例。借助思维链方法，我们将指代任务分解为多个可解释的中间步骤，既提升了性能，也让框架具备极高的数据效率。此外，所提框架可轻松集成到任意现有架构中。我们在Nr3D、Sr3D和Scanrefer基准上开展了全面实验验证，结果表明，在无需手动标注数据的情况下，该方法相比现有方法能带来持续的性能提升。我们将该框架命名为CoT3DRef，其数据效率优势显著：在Sr3D数据集上，仅用10%的训练数据就能达到使用全量数据训练的SOTA（当前最优）性能。代码可在https://eslambakr.github.io/cot3dref.github.io/获取。
英文摘要：
3D visual grounding is the ability to localize objects in 3D scenes conditioned by utterances. Most existing methods devote the referring head to localize the referred object directly, causing failure in complex scenarios. In addition, it does not illustrate how and why the network reaches the final decision. In this paper, we address this question Can we design an interpretable 3D visual grounding framework that has the potential to mimic the human perception system?. To this end, we formulate the 3D visual grounding problem as a sequence-to-sequence Seq2Seq task by first predicting a chain of anchors and then the final target. Interpretability not only improves the overall performance but also helps us identify failure cases. Following the chain of thoughts approach enables us to decompose the referring task into interpretable intermediate steps, boosting the performance and making our framework extremely data-efficient. Moreover, our proposed framework can be easily integrated into any existing architecture. We validate our approach through comprehensive experiments on the Nr3D, Sr3D, and Scanrefer benchmarks and show consistent performance gains compared to existing methods without requiring manually annotated data. Furthermore, our proposed framework, dubbed CoT3DRef, is significantly data-efficient, whereas on the Sr3D dataset, when trained only on 10% of the data, we match the SOTA performance that trained on the entire data. The code is available at https:eslambakr.github.io/cot3dref.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
