3D CoCa：对比学习器是3D描述生成器
3D CoCa: Contrastive Learners are 3D Captioners
AI总结：
针对3D描述生成中点云稀疏性与跨模态对齐弱的问题，提出3D CoCa统一框架，联合对比视觉-语言学习与描述生成，在ScanRefer和Nr3D基准的CIDEr指标上分别提升10.2%和5.76%。
摘要：
3D描述生成旨在以自然语言描述3D场景内容，但由于点云固有的稀疏性以及现有方法中跨模态对齐较弱，该任务仍极具挑战性。为解决这些挑战，我们提出3D CoCa，这是一种新颖的统一框架，能在单一架构中无缝结合对比视觉-语言学习与3D描述生成。我们的方法利用冻结的CLIP视觉-语言骨干网络提供丰富语义先验，通过空间感知3D场景编码器捕捉几何上下文，并使用多模态解码器生成描述性文本。与依赖显式目标提议的现有两阶段方法不同，3D CoCa在共享特征空间中联合优化对比和描述生成目标，无需外部检测器或手工提议。这种联合训练范式通过对齐3D与文本表示，实现更强的空间推理与更丰富的语义接地。在ScanRefer和Nr3D基准上的大量实验表明，3D CoCa在0.5IoU下的CIDEr指标分别显著优于当前最先进方法10.2%和5.76%。代码将在https://github.com/AIGeeksGroup/3DCoCa发布。
英文摘要：
3D captioning, which aims to describe the content of 3D scenes in natural language, remains highly challenging due to the inherent sparsity of point clouds and weak cross-modal alignment in existing methods. To address these challenges, we propose 3D CoCa, a novel unified framework that seamlessly combines contrastive vision-language learning with 3D caption generation in a single architecture. Our approach leverages a frozen CLIP vision-language backbone to provide rich semantic priors, a spatially-aware 3D scene encoder to capture geometric context, and a multi-modal decoder to generate descriptive captions. Unlike prior two-stage methods that rely on explicit object proposals, 3D CoCa jointly optimizes contrastive and captioning objectives in a shared feature space, eliminating the need for external detectors or handcrafted proposals. This joint training paradigm yields stronger spatial reasoning and richer semantic grounding by aligning 3D and textual representations. Extensive experiments on the ScanRefer and Nr3D benchmarks demonstrate that 3D CoCa significantly outperforms current state-of-the-arts by 10.2% and 5.76% in CIDEr at 0.5IoU, respectively. Code will be available at https://github.com/AIGeeksGroup/3DCoCa.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
