评估零样本GPT-4V在三维视觉问答基准上的性能
Evaluating Zero-Shot GPT-4V Performance on 3D Visual Question Answering Benchmarks
AI总结：
本研究评估GPT-4V和GPT-4在3D VQA基准上的零样本性能，发现无需微调即可与封闭词汇方法持平，且场景特定词汇能显著提升智能体表现。
摘要：
随着人们对在基础模型背景下“重构”三维视觉问答（3D VQA）问题的兴趣日益增长，评估这些新范式如何影响现有的封闭词汇数据集变得至关重要。在本案例研究中，我们评估了基础模型（GPT-4 Vision和GPT-4）在成熟的3D VQA基准（即3D-VQA和ScanQA）上的零样本性能。我们提供了一项调查，以将基于GPT的智能体相对于传统建模方法的性能进行情境化分析。我们发现，未经任何微调的基于GPT的智能体与封闭词汇方法的性能相当。我们的研究结果证实了最近的研究结论，即“盲”模型在封闭词汇环境中建立了令人惊讶的强基线。我们证明，智能体通过上下文文本接地从场景特定词汇中显著受益。通过提供与先前基线的初步比较，我们希望为社区完善多模态3D基准的持续努力提供信息。
英文摘要：
As interest in "reformulating" the 3D Visual Question Answering (VQA) problem in the context of foundation models grows, it is imperative to assess how these new paradigms influence existing closed-vocabulary datasets. In this case study, we evaluate the zero-shot performance of foundational models (GPT-4 Vision and GPT-4) on well-established 3D VQA benchmarks, namely 3D-VQA and ScanQA. We provide an investigation to contextualize the performance of GPT-based agents relative to traditional modeling approaches. We find that GPT-based agents without any fine-tuning perform on par with the closed vocabulary approaches. Our findings corroborate recent results that "blind" models establish a surprisingly strong baseline in closed-vocabulary settings. We demonstrate that agents benefit significantly from scene-specific vocabulary via in-context textual grounding. By presenting a preliminary comparison with previous baselines, we hope to inform the community's ongoing efforts to refine multi-modal 3D benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
