GeoBench: 通过分层评估重新思考多模态几何问题解决
GeoBench: Rethinking Multimodal Geometric Problem-Solving via Hierarchical Evaluation
AI总结：
GeoBench通过分层评估框架，系统评估几何问题解决能力，揭示任务复杂度对性能的影响及子目标分解的重要性。
摘要：
几何问题解决是数学推理的重要分支，需要对形状和空间关系进行精确分析。当前对视觉语言模型（VLMs）中几何推理的评估面临局限性，包括测试数据可能受到基于教科书的基准测试污染的风险，过度强调最终答案而非推理过程，以及诊断粒度不足的问题。为了解决这些问题，我们提出了GeoBench，一个具有四个几何问题解决推理层次的分层基准：视觉感知、目标导向规划、严谨定理应用和自我反思回溯。通过六个经过正式验证的任务生成器TrustGeoGen生成的任务，我们系统地评估了从属性提取到逻辑错误纠正的能力。实验发现，虽然推理模型如OpenAI-o3在一般MLLMs上表现更优，但随着任务复杂度的增加，性能显著下降。关键发现表明，子目标分解和无关前提过滤对最终问题解决准确性有关键影响，而思维链提示法在某些任务中意外降低了性能。这些发现使GeoBench成为全面的基准，同时为开发几何问题解决系统提供了可操作的指导。
英文摘要：
Geometric problem solving constitutes a critical branch of mathematical reasoning, requiring precise analysis of shapes and spatial relationships. Current evaluations of geometric reasoning in vision-language models (VLMs) face limitations, including the risk of test data contamination from textbook-based benchmarks, overemphasis on final answers over reasoning processes, and insufficient diagnostic granularity. To address these issues, we present GeoBench, a hierarchical benchmark featuring four reasoning levels in geometric problem-solving: Visual Perception, Goal-Oriented Planning, Rigorous Theorem Application, and Self-Reflective Backtracking. Through six formally verified tasks generated via TrustGeoGen, we systematically assess capabilities ranging from attribute extraction to logical error correction. Experiments reveal that while reasoning models like OpenAI-o3 outperform general MLLMs, performance declines significantly with increasing task complexity. Key findings demonstrate that sub-goal decomposition and irrelevant premise filtering critically influence final problem-solving accuracy, whereas Chain-of-Thought prompting unexpectedly degrades performance in some tasks. These findings establish GeoBench as a comprehensive benchmark while offering actionable guidelines for developing geometric problem-solving systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
