多语言视觉语言模型是否同样具备推理能力？一种针对印度语言的跨语言视觉推理审计
Do Multilingual VLMs Reason Equally? A Cross-Lingual Visual Reasoning Audit for Indian Languages
AI总结：
本文通过将MathVista、ScienceQA和MMMU的980道题翻译成印度语言，评估了8种VLM在七种语言中的表现，发现切换至印度语言时准确率下降9.8-25个百分点，且达罗毗荼语比印欧语下降更多，链式推理反而降低了部分语言的性能。
摘要：
视觉语言模型在数学、科学和空间推理基准测试中表现良好，但这些评估大多以英语为主。本文首次针对印度语言进行跨语言视觉推理审计。使用IndicTrans2将MathVista、ScienceQA和MMMU的980道题翻译成印地语、泰米尔语、泰卢固语、孟加拉语、卡纳达语和马拉地语，通过Gemini 2.0 Flash对50个样本进行交叉验证（译者间一致性0.79-0.84）。在所有七种语言上评估了八种VLM，从7B开源模型到GPT-4o，产生68,600条推理记录，包括纯文本和链式推理消融。发现从英语切换到印度语言时准确率下降9.8-25个百分点，达罗毗荼语比印欧语下降更多。链式推理提示反而降低了孟加拉语（-14.4个百分点）和卡纳达语（-11.4个百分点）的性能，暴露了以英语为中心的推理链。Aya-Vision-8B在23种语言中构建，但在达罗毗荼语中仍下降28.5个百分点；多语言预训练本身不足以转移视觉推理能力。本文发布翻译后的基准测试和所有模型输出。
英文摘要：
Vision-language models score well on mathematical, scientific, and spatial reasoning benchmarks, yet these evaluations are overwhelmingly English. I present the first cross-lingual visual reasoning audit for Indian languages. 980 questions from MathVista, ScienceQA, and MMMU are translated into Hindi, Tamil, Telugu, Bengali, Kannada, and Marathi using IndicTrans2, with Gemini 2.0 Flash cross-verification on 50 samples per language (inter-translator agreement 0.79-0.84). Eight VLMs, from 7B open-source models to GPT-4o, are evaluated across all seven languages, yielding 68,600 inference records that include text-only and chain-of-thought ablations. I find accuracy drops of 9.8-25 percentage points when switching from English to an Indian language, with Dravidian languages suffering up to 13.2 pp more than Indo-Aryan. Chain-of-thought prompting degrades Bengali (-14.4 pp) and Kannada (-11.4 pp) rather than helping, exposing English-centric reasoning chains. Aya-Vision-8B, built for 23 languages, still drops 28.5 pp on Dravidian scripts; multilingual pretraining alone does not transfer visual reasoning. I release the translated benchmark and all model outputs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
