arXiv 2505.150752025-08-24cs.CLcs.AIcs.CVcs.LG

跨语言之旅：多模态大语言模型跨语言一致性基准测试

Traveling Across Languages: Benchmarking Cross-Lingual Consistency in Multimodal LLMs

Hao Wang, Pinzhi Huang, Jihan Yang, Saining Xie, Daisuke Kawahara

备注：: The first version of this paper mistakenly included a prompt injection phrase, which was inappropriate and unprofessional. Although we corrected the version on arXiv and withdrew from the conference, my co-authors and university strongly request a full withdrawal. Given the situation, I no longer have the authority to manage this paper, and withdrawing it from arXiv is the most responsible action

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对多模态大语言模型（MLLMs）跨语言性能不一致问题，本文提出KnowRecall和VisRecall两个基准，分别评估15种语言的事实知识一致性与9种语言的视觉记忆一致性，实验表明现有MLLMs仍存在挑战，为相关研究提供评估工具。

摘要：

多模态大语言模型（MLLMs）的快速发展显著增强了其现实应用。然而，在跨语言间实现一致性能，尤其是整合文化知识时，仍面临重大挑战。为更好评估该问题，我们引入KnowRecall和VisRecall两个新基准，以评估MLLMs的跨语言一致性。KnowRecall是视觉问答基准，旨在衡量15种语言的事实知识一致性，聚焦全球地标相关的文化历史问题；VisRecall通过要求模型在无图像时描述地标外观，评估视觉记忆一致性。实验结果显示，包括专有模型在内的最先进MLLMs在跨语言一致性方面仍存在困难，凸显了对更稳健方法的需求，以打造真正多语言且具文化感知的模型。

英文摘要：

The rapid evolution of multimodal large language models (MLLMs) has significantly enhanced their real-world applications. However, achieving consistent performance across languages, especially when integrating cultural knowledge, remains a significant challenge. To better assess this issue, we introduce two new benchmarks: KnowRecall and VisRecall, which evaluate cross-lingual consistency in MLLMs. KnowRecall is a visual question answering benchmark designed to measure factual knowledge consistency in 15 languages, focusing on cultural and historical questions about global landmarks. VisRecall assesses visual memory consistency by asking models to describe landmark appearances in 9 languages without access to images. Experimental results reveal that state-of-the-art MLLMs, including proprietary ones, still struggle to achieve cross-lingual consistency. This underscores the need for more robust approaches that produce truly multilingual and culturally aware models.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。