关注（语言）鸿沟：探索LVLMs的数值与跨语言能力极限
Mind the (Language) Gap: Towards Probing Numerical and Cross-Lingual Limits of LVLMs
AI总结：
该研究构建板球记分卡视觉问答基准MMCRICBENCH-3K，评估LVLM的复杂数值与跨语言推理能力，揭示其在结构感知、数值推理和跨语言泛化方面的局限。
摘要：
我们引入MMCRICBENCH-3K，这是一个针对板球记分卡的视觉问答（VQA）基准，旨在评估大型视觉语言模型（LVLMs）对半结构化表格图像的复杂数值及跨语言推理能力。该基准包含1463张合成生成的ODI、T20和Test格式的记分卡图像，以及1500组英文问答对，分为两个子集：含英文记分卡的MMCRICBENCH-E-1.5K和含视觉相似印地语记分卡的MMCRICBENCH-H-1.5K，所有问答均为英文以实现受控跨脚本评估。任务要求对结构化数值数据、多图像上下文和隐含领域知识进行推理。实验结果显示，即使是GPT-4o和Qwen2.5VL等最先进的LVLM，在其主要训练语言的英文子集上仍表现不佳，且在印地语子集上性能进一步下降，这揭示了它们在结构感知视觉文本理解、数值推理和跨语言泛化方面的关键局限。该数据集可通过Hugging Face公开获取（https://huggingface.co/datasets/DIALab/MMCricBench），以推动该方向的LVLM研究。
英文摘要：
We introduce MMCRICBENCH-3K, a benchmark for Visual Question Answering (VQA) on cricket scorecards, designed to evaluate large vision-language models (LVLMs) on complex numerical and cross-lingual reasoning over semi-structured tabular images. MMCRICBENCH-3K comprises 1,463 synthetically generated scorecard images from ODI, T20, and Test formats, accompanied by 1,500 English QA pairs. It includes two subsets: MMCRICBENCH-E-1.5K, featuring English scorecards, and MMCRICBENCH-H-1.5K, containing visually similar Hindi scorecards, with all questions and answers kept in English to enable controlled cross-script evaluation. The task demands reasoning over structured numerical data, multi-image context, and implicit domain knowledge. Empirical results show that even state-of-the-art LVLMs, such as GPT-4o and Qwen2.5VL, struggle on the English subset despite it being their primary training language and exhibit a further drop in performance on the Hindi subset. This reveals key limitations in structure-aware visual text understanding, numerical reasoning, and cross-lingual generalization. The dataset is publicly available via Hugging Face at https://huggingface.co/datasets/DIALab/MMCricBench, to promote LVLM research in this direction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
