SpatialClaw：重新思考智能体空间推理的动作接口
SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning
AI总结：
提出SpatialClaw框架，以代码作为动作接口，通过状态化Python内核和感知几何原语，使VLM智能体逐步执行并灵活组合中间结果，在20个3D/4D空间推理基准上平均准确率59.9%，比现有方法高11.2个百分点。
摘要：
空间推理——确定物体在3D空间中的位置、关系及运动方式的能力——仍然是视觉语言模型（VLM）面临的基本挑战。工具增强型智能体试图通过为VLM添加专业感知模块来解决这一问题，但其有效性受限于调用这些工具的动作接口。本文研究该接口的设计如何影响智能体进行开放式空间推理的能力。现有的空间智能体要么采用单次代码执行，即在观察到任何中间结果之前就确定完整的分析策略；要么依赖结构化的工具调用接口，这通常缺乏自由组合操作或针对每个任务定制分析的灵活性。这两种设计对开放式、复杂的3D/4D空间推理的灵活性有限。因此，我们提出SpatialClaw，一个无需训练的空间推理框架，采用代码作为动作接口。SpatialClaw维护一个状态化的Python内核，预加载输入帧和一套感知与几何原语，让基于VLM的智能体在每一步根据所有先前输出编写一个可执行单元，从而灵活地组合和操作感知结果，并根据中间文本和视觉观察以及每个问题的需求调整其分析。在涵盖广泛静态和动态3D/4D空间推理任务的20个空间推理基准上评估，SpatialClaw实现了59.9%的平均准确率，比最新的空间智能体高出11.2个百分点，并且在来自两个模型家族的六个VLM骨干网络上均取得一致提升，无需任何基准或模型特定的适配。
英文摘要：
Spatial reasoning, the ability to determine where objects are, how they relate, and how they move in 3D, remains a fundamental challenge for vision-language models (VLMs). Tool-augmented agents attempt to address this by augmenting VLMs with specialist perception modules, yet their effectiveness is bounded by the action interface through which those tools are invoked. In this work, we study how the design of this interface shapes the agent's capacity for open-ended spatial reasoning. Existing spatial agents either employ single-pass code execution, which commits to a full analysis strategy before any intermediate result is observed, or rely on a structured tool-call interface that often offers less flexibility for freely composing operations or tailoring the analysis to each task. Both designs offer limited flexibility for open-ended, complex 3D/4D spatial reasoning. We therefore propose SpatialClaw, a training-free framework for spatial reasoning that adopts code as the action interface. SpatialClaw maintains a stateful Python kernel pre-loaded with input frames and a suite of perception and geometry primitives, letting a VLM-backed agent write one executable cell per step conditioned on all prior outputs, enabling the agent to flexibly compose and manipulate perception results and adapt its analysis to both intermediate text and visual observations and the demands of each problem. Evaluated across 20 spatial reasoning benchmarks spanning a broad range of static and dynamic 3D/4D spatial reasoning tasks, SpatialClaw achieves 59.9% average accuracy, outperforming the recent spatial agent by +11.2 points, with consistent gains across six VLM backbones from two model families without any benchmark- or model-specific adaptation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
