MathSight: 一个探索视觉语言模型在大学级数学推理中是否真正看到的基准
MathSight: A Benchmark Exploring Have Vision-Language Models Really Seen in University-Level Mathematical Reasoning?
AI总结：
MathSight基准测试通过对比不同视觉输入条件，探索视觉语言模型在大学级数学推理中视觉信息的真实贡献。
摘要：
最近视觉语言模型（VLMs）在多模态数学推理方面取得了显著进展。然而，有多少视觉信息真正有助于推理仍不清楚。现有基准测试报告出总体表现强劲，但很少孤立图像模态的作用，留下开放性问题：VLMs是否真正利用视觉理解，还是仅仅依赖语言先验。为此，我们提出了MathSight，一个大学级的多模态数学推理基准，旨在解构和量化视觉输入的影响。每个问题包含多个视觉变体--原始、手绘、照片捕捉--以及一个纯文本条件用于受控比较。在最先进的VLMs上的实验揭示了一种一致的趋势：随着问题难度的增加，视觉信息的贡献逐渐减少。值得注意的是，没有任何图像输入的Qwen3-VL超越了其多模态变体和GPT-5，凸显了如MathSight这样的基准测试在未来模型中推进真正的视觉基础推理的必要性。
英文摘要：
Recent advances in Vision-Language Models (VLMs) have achieved impressive progress in multimodal mathematical reasoning. Yet, how much visual information truly contributes to reasoning remains unclear. Existing benchmarks report strong overall performance but seldom isolate the role of the image modality, leaving open whether VLMs genuinely leverage visual understanding or merely depend on linguistic priors. To address this, we present MathSight, a university-level multimodal mathematical reasoning benchmark designed to disentangle and quantify the effect of visual input. Each problem includes multiple visual variants -- original, hand-drawn, photo-captured -- and a text-only condition for controlled comparison. Experiments on state-of-the-art VLMs reveal a consistent trend: the contribution of visual information diminishes with increasing problem difficulty. Remarkably, Qwen3-VL without any image input surpasses both its multimodal variants and GPT-5, underscoring the need for benchmarks like MathSight to advance genuine vision-grounded reasoning in future models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
