SQA3D：3D场景中的情境化问答
SQA3D: Situated Question Answering in 3D Scenes
AI总结：
本文提出SQA3D任务与数据集，要求智能体在3D场景中基于文本描述的情境进行推理问答，并评估现有模型与人类表现差距，以推动具身AI的情境理解与推理研究。
摘要：
我们提出一项新任务，用于基准测试具身智能体的场景理解能力：3D场景中的情境化问答（SQA3D）。给定一个场景上下文（例如3D扫描），SQA3D要求被测试的智能体首先理解其在3D场景中由文本描述的情境（位置、朝向等），然后推理其周围环境并在该情境下回答问题。基于ScanNet中的650个场景，我们提供了一个以约6.8k个独特情境为中心的数据集，并包含20.4k条描述和33.4k个针对这些情境的多样化推理问题。这些问题考察智能体广泛的推理能力，涵盖空间关系理解、常识理解、导航和多跳推理。SQA3D对当前多模态尤其是3D推理模型构成了重大挑战。我们评估了各种最先进的方法，发现最佳方法仅取得47.20%的总体得分，而业余人类参与者可达到90.06%。我们相信SQA3D能够促进未来具身AI研究，使其具备更强的情境理解和推理能力。
英文摘要：
We propose a new task to benchmark scene understanding of embodied agents: Situated Question Answering in 3D Scenes (SQA3D). Given a scene context (e.g., 3D scan), SQA3D requires the tested agent to first understand its situation (position, orientation, etc.) in the 3D scene as described by text, then reason about its surrounding environment and answer a question under that situation. Based upon 650 scenes from ScanNet, we provide a dataset centered around 6.8k unique situations, along with 20.4k descriptions and 33.4k diverse reasoning questions for these situations. These questions examine a wide spectrum of reasoning capabilities for an intelligent agent, ranging from spatial relation comprehension to commonsense understanding, navigation, and multi-hop reasoning. SQA3D imposes a significant challenge to current multi-modal especially 3D reasoning models. We evaluate various state-of-the-art approaches and find that the best one only achieves an overall score of 47.20%, while amateur human participants can reach 90.06%. We believe SQA3D could facilitate future embodied AI research with stronger situation understanding and reasoning capability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
