M4-RAG：大规模多语言多文化多模态检索增强生成
M4-RAG: A Massive-Scale Multilingual Multi-Cultural Multimodal RAG
AI总结：
M4-RAG提出一个覆盖42种语言、56种方言和189个国家的多模态大规模基准，通过构建8万多个文化多样化的图像-问题对，评估跨语言和模态的检索增强视觉问答性能，揭示模型大小与检索效果的不匹配问题。
摘要：
视觉语言模型（VLMs）在视觉问答（VQA）中表现出色，但受限于静态训练数据。检索增强生成（RAG）通过允许访问最新、文化相关和多语言信息来缓解这一限制；然而，多语言多模态RAG仍鲜有研究。我们引入M4-RAG，一个覆盖42种语言、56种方言和189个国家的大规模基准，包含超过8万组文化多样的图像-问题对，用于评估跨语言和模态的检索增强VQA。为平衡现实与可重复性，我们构建了一个受控检索环境，包含数百万精心挑选的多语言文档，近似真实世界检索条件，同时确保一致的实验。我们的系统评估表明，尽管RAG对小型VLMs有益，但无法扩展到大型模型，甚至会降低其性能，揭示了模型大小与当前检索效果之间的关键不匹配。我们的跨语言评估还显示，当提示或检索上下文提供非英语语言时，性能显著下降。M4-RAG的代码、数据集和评估协议已作为开源在https://github.com/davidanugraha/M4-RAG提供。
英文摘要：
Vision-language models (VLMs) have achieved strong performance in visual question answering (VQA), yet they remain constrained by static training data. Retrieval-Augmented Generation (RAG) mitigates this limitation by enabling access to up-to-date, culturally grounded, and multilingual information; however, multilingual multimodal RAG remains largely underexplored. We introduce M4-RAG, a massive-scale benchmark spanning 42 languages, 56 regional dialects and registers, and 189 countries, comprising over 80,000 culturally diverse image-question pairs for evaluating retrieval-augmented VQA across languages and modalities. To balance realism with reproducibility, we build a controlled retrieval environment containing millions of carefully curated multilingual documents relevant to the query domains, approximating real-world retrieval conditions while ensuring consistent experimentation. Our systematic evaluation reveals that although RAG consistently benefits smaller VLMs, it fails to scale to larger models and often even degrades their performance, exposing a critical mismatch between model size and current retrieval effectiveness. Our cross-lingual evaluations also reveal significant performance degradation when prompts or retrieved context are provided in non-English languages. The code, datasets, and evaluation protocols for M4-RAG are available as open-source at https://github.com/davidanugraha/M4-RAG.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
