多语言视觉语言模型训练：将英语训练的VLM适应到法语
Multilingual VLM Training: Adapting an English-Trained VLM to French
AI总结：
本文研究了将英语训练的视觉语言模型适应到法语等其他语言的挑战，通过比较不同方法的性能和计算成本，发现数据翻译是多语言VLM性能的主要瓶颈。
摘要：
人工智能近年来取得了巨大进展，特别是在视觉-语言模型（VLMs）的发展方面，这些模型能够理解和处理视觉和文本数据。然而，这些进步仍然主要局限于英语，限制了非英语使用者的可及性。有必要将这些能力扩展到更广泛的语言。本文探讨了将英语训练的VLM适应到不同语言所面临的挑战。为此，我们将探索并比较不同方法在性能和计算成本方面的表现。我们考虑了基于翻译的流程、LoRA微调以及一种两阶段微调策略，该策略将视觉适应与语言适应分开。为了评估这些方法，我们使用了将标准多模态基准转换为目标语言的组合以及由母语专家进行的手动评估。结果表明，数据翻译仍然是多语言VLM性能的主要瓶颈，数据质量限制了训练和评估的有效性。这些发现表明，未来的工作应专注于母语数据集的收集和改进的翻译策略。
英文摘要：
Artificial intelligence has made great progress in recent years, particularly in the development of Vision--Language Models (VLMs) that understand both visual and textual data. However, these advancements remain largely limited to English, reducing their accessibility for non--English speakers. It is essential to extend these capabilities to a broader range of languages. This paper explores the challenges of adapting an English-trained VLM to different languages. To this end, we will explore and compare different methods for their performance and computational cost. We consider a translation-based pipeline, LoRA finetuning, and a two-stage finetuning strategy that separates vision adaptation from language adaptation. To evaluate these methods, we use a combination of standard multimodal benchmarks translated into the target language and manual assessments by native experts. The results reveal that dataset translation remains a major bottleneck in multilingual VLM performance, with data quality limiting the effectiveness of training and evaluation. These findings suggest that future efforts should focus on native-language dataset collection and improved translation strategies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
