3D-MoRe：面向具身问答的统一模态-上下文推理
3D-MoRe: Unified Modal-Contextual Reasoning for Embodied Question Answering
AI总结：
提出3D-MoRe框架，利用基础模型生成大规模3D语言数据集，在ScanQA和ScanRefer上分别提升CIDEr 2.15%和CIDEr@0.5 1.84%，并公开代码与数据。
摘要：
随着室内场景任务（如问答和密集描述）对多样化和可扩展数据的需求日益增长，我们提出了3D-MoRe，一种新颖的范式，旨在通过利用基础模型的优势来生成大规模3D语言数据集。该框架整合了关键组件，包括多模态嵌入、跨模态交互和语言模型解码器，以处理自然语言指令和3D场景数据。这种方法促进了在复杂3D环境中的增强推理和响应生成。使用ScanNet 3D场景数据集以及ScanQA和ScanRefer的文本标注，3D-MoRe在1,513个场景中生成了62,000个问答（QA）对和73,000个物体描述。我们还采用了各种数据增强技术，并实施了语义过滤以确保数据质量。在ScanQA上的实验表明，3D-MoRe显著优于最先进的基线，CIDEr分数提高了2.15%。同样，在ScanRefer上，我们的方法在CIDEr@0.5上实现了1.84%的显著提升，突显了其在两个任务中的有效性。我们的代码和生成的数据集将公开发布以惠及社区，两者均可通过https://3D-MoRe.github.io访问。
英文摘要：
With the growing need for diverse and scalable data in indoor scene tasks, such as question answering and dense captioning, we propose 3D-MoRe, a novel paradigm designed to generate large-scale 3D-language datasets by leveraging the strengths of foundational models. The framework integrates key components, including multi-modal embedding, cross-modal interaction, and a language model decoder, to process natural language instructions and 3D scene data. This approach facilitates enhanced reasoning and response generation in complex 3D environments. Using the ScanNet 3D scene dataset, along with text annotations from ScanQA and ScanRefer, 3D-MoRe generates 62,000 question-answer (QA) pairs and 73,000 object descriptions across 1,513 scenes. We also employ various data augmentation techniques and implement semantic filtering to ensure high-quality data. Experiments on ScanQA demonstrate that 3D-MoRe significantly outperforms state-of-the-art baselines, with the CIDEr score improving by 2.15\%. Similarly, on ScanRefer, our approach achieves a notable increase in CIDEr@0.5 by 1.84\%, highlighting its effectiveness in both tasks. Our code and generated datasets will be publicly released to benefit the community, and both can be accessed on the https://3D-MoRe.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
