Flame3D: 无需3D特定训练的3D场景零样本组合推理
Flame3D: Zero-shot Compositional Reasoning of 3D Scenes with Agentic Language Models
AI总结：
Flame3D通过可组合的空间工具和训练自由框架，实现3D场景的零样本组合推理，支持动态空间合成和外部数据整合，展示了在ScanQA和Compose3D上的竞争力。
摘要：
3D场景理解涉及自由空间推理、物体定位、假设物体插入、复杂几何关系及与外部工具和数据源的整合。现有方法通常依赖大规模3D语言训练或聚焦于物体定位和简单空间关系。本文提出Flame3D，一种无需3D特定训练的框架，将场景表示为可编辑的视觉-文本3D记忆，并通过可组合的空间工具暴露给现成的MLLM。Flame3D还允许代理在推理时合成自定义空间程序，实现对布局、空闲空间和尚未出现在场景中的物体的开放推理。外部数据和修正可添加到记忆中而无需重新训练。除了在ScanQA上与微调3D-LMM方法竞争性表现外，我们通过在定制的组合空间推理基准Compose3D上评估Flame3D的多跳3D推理能力，发现固定工具不足，而代理在推理时合成空间操作的能力至关重要。这些结果引发问题：未来3D场景理解的进步应聚焦于更丰富的场景记忆和表达性的组合抽象吗？
英文摘要：
3D scene understanding spans reasoning about free space, object grounding, hypothetical object insertions, complex geometric relationships, and integrating all of these with external tools and data sources. Existing 3D understanding methods typically rely on large-scale 3D-language training or focus on object grounding and simple spatial relationships. We argue that the broad generalization that motivates 3D-language training can be achieved at inference time, without 3D-specific training. We propose Flame3D, a training-free framework that represents scenes as editable visual-textual 3D memories and exposes them to an off-the-shelf MLLM through composable spatial tools. Flame3D also lets the agent synthesize custom spatial programs at inference time, enabling open-ended reasoning over layouts, empty space, and objects not yet present in the scene. External data and corrections can be added to the memory without retraining. In addition to showing competitive performance to finetuned 3D-LMM methods on ScanQA, we study multi-hop 3D reasoning capabilities of Flame3D by evaluating it on a curated compositional spatial-reasoning benchmark, Compose3D. We find that fixed tools fall short and that the agent's ability to synthesize spatial operations at inference time is essential. These results invite the question: should future progress in 3D scene understanding focus on richer scene memories and expressive compositional abstractions?
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
