DynaSolidGeo：用于评估视觉语言模型在立体几何中真实空间数学推理能力的动态基准
DynaSolidGeo: A Dynamic Benchmark for Genuine Spatial Mathematical Reasoning of VLMs in Solid Geometry
AI总结：
针对现有多模态数学推理基准聚焦2D几何、依赖静态数据集易污染且忽略推理过程的局限，本文提出首个动态基准DynaSolidGeo，含503个种子问题可动态生成实例并引入过程评估，实验揭示VLMs在立体几何空间推理中的性能短板。
摘要：
立体几何问题求解需要整合空间智能与符号推理的空间数学推理能力。然而，大多数现有多模态数学推理基准主要聚焦于2D平面几何，依赖易受数据污染和记忆影响的静态数据集，且仅通过最终答案评估模型，忽略推理过程。为解决这些局限，我们引入DynaSolidGeo，这是首个用于评估视觉语言模型（VLMs）真实空间推理能力的动态基准。通过半自动标注流水线构建，DynaSolidGeo包含503个专家精选的种子问题，原则上可动态生成无限数量的多样化多模态文本-视觉实例。除答案准确性外，我们还基于专家标注的推理链引入过程评估，以衡量逻辑有效性和因果连贯性。对代表性开源与闭源VLMs的实验显示，模型间存在巨大性能差距，在动态设置下性能严重下降，且在需要高级空间智能（如心理旋转和可视化）的任务上表现不佳。代码和数据集可在DynaSolidGeo获取。
英文摘要：
Solid geometry problem solving demands spatial mathematical reasoning that integrates spatial intelligence and symbolic reasoning. However, most existing multimodal mathematical reasoning benchmarks focus primarily on 2D plane geometry, rely on static datasets prone to data contamination and memorization, and evaluate models solely by final answers, overlooking the reasoning process. To address these limitations, we introduce DynaSolidGeo, the first dynamic benchmark for evaluating genuine spatial reasoning in Vision-Language Models (VLMs). Constructed through a semi-automatic annotation pipeline, DynaSolidGeo contains 503 expert-curated seed questions that can, in principle, dynamically generate an unbounded number of diverse multimodal text-visual instances. Beyond answer accuracy, we incorporate process evaluation based on expert-annotated reasoning chains to measure logical validity and causal coherence. Experiments across representative open-source and closed-source VLMs reveal large performance gaps, severe degradation in dynamic settings, and poor performance on tasks requiring high-level spatial intelligence, such as mental rotation and visualization. The code and dataset are available at \href{https://zgca-ai4edu.github.io/DynaSolidGeo/}{DynaSolidGeo}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
