Rice-VL：评估跨东盟国家的视觉语言模型的文化理解能力
Rice-VL: Evaluating Vision-Language Models for Cultural Understanding Across ASEAN Countries
AI总结：
RICE-VL通过评估视觉语言模型在11个东盟国家的文化理解能力，揭示了模型在文化多样性地区存在的偏见和局限性，强调了开发更具包容性的模型的重要性。
摘要：
视觉语言模型（VLMs）在多模态任务中表现出色，但往往存在以西方为中心的偏见，限制了其在文化多样性的地区如东南亚（SEA）的有效性。为此，我们引入了RICE-VL，一个新颖的基准，用于评估VLM在11个东盟国家之间的文化理解能力。RICE-VL包含超过28,000个由人类精心编写的视觉问答（VQA）样本——涵盖真假判断、填空和开放式格式，并包含1,000对图像边界框用于视觉定位，由具有文化意识的专家在14个子定位类别中注释。我们提出了SEA-LAVE，LAVE指标的扩展，用于评估文本准确性、文化契合度和国家识别。对六个开源和闭源VLM的评估揭示了在低资源国家和抽象文化领域存在显著性能差距。视觉定位任务测试了模型在复杂场景中定位文化重要元素的能力，探测了空间和上下文准确性。RICE-VL暴露了VLM在文化理解方面的局限性，并突显了需要包容性模型开发以更好地服务全球多样化人群的必要性。
英文摘要：
Vision-Language Models (VLMs) excel in multimodal tasks but often exhibit Western-centric biases, limiting their effectiveness in culturally diverse regions like Southeast Asia (SEA). To address this, we introduce RICE-VL, a novel benchmark evaluating VLM cultural understanding across 11 ASEAN countries. RICE-VL includes over 28,000 human-curated Visual Question Answering (VQA) samples -- covering True or False, Fill-in-the-Blank, and open-ended formats -- and 1,000 image-bounding box pairs for Visual Grounding, annotated by culturally informed experts across 14 sub-ground categories. We propose SEA-LAVE, an extension of the LAVE metric, assessing textual accuracy, cultural alignment, and country identification. Evaluations of six open- and closed-source VLMs reveal significant performance gaps in low-resource countries and abstract cultural domains. The Visual Grounding task tests models' ability to localize culturally significant elements in complex scenes, probing spatial and contextual accuracy. RICE-VL exposes limitations in VLMs' cultural comprehension and highlights the need for inclusive model development to better serve diverse global populations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
