m3P：基于多模态提示的多模态多语言翻译
m3P: Towards Multimodal Multilingual Translation with Multimodal Prompt
AI总结：
本文提出m3P框架，利用多模态提示引导多模态多语言神经机器翻译，通过引入视觉上下文作为语言无关表示，以图像为中心语言最小化不同语言表示距离，构建支持102种语言的InstrMulti102数据集，大幅超越基线方法。
摘要：
多语言翻译通过将所有语言投影到一个共享空间来支持多个翻译方向，但在纯文本模态下，语言之间的差异会削弱翻译质量，尤其是在语言数量庞大时。为了弥合这一差距，我们引入视觉上下文作为通用的语言无关表示，以促进多语言翻译。在本文中，我们提出了一个框架，利用多模态提示来引导多模态多语言神经机器翻译（m3P），该模型对齐具有相同含义的不同语言表示，并生成用于翻译的条件视觉-语言记忆。我们构建了一个多语言多模态指令数据集（InstrMulti102）来支持102种语言。我们的方法旨在通过将图像作为中心语言，最小化不同语言的表示距离。实验结果表明，m3P大幅优于之前的纯文本基线和多语言多模态方法。此外，探测实验验证了我们的方法在低资源和大规模多语言场景下增强翻译的有效性。
英文摘要：
Multilingual translation supports multiple translation directions by projecting all languages in a shared space, but the translation quality is undermined by the difference between languages in the text-only modality, especially when the number of languages is large. To bridge this gap, we introduce visual context as the universal language-independent representation to facilitate multilingual translation. In this paper, we propose a framework to leverage the multimodal prompt to guide the Multimodal Multilingual neural Machine Translation (m3P), which aligns the representations of different languages with the same meaning and generates the conditional vision-language memory for translation. We construct a multilingual multimodal instruction dataset (InstrMulti102) to support 102 languages. Our method aims to minimize the representation distance of different languages by regarding the image as a central language. Experimental results show that m3P outperforms previous text-only baselines and multilingual multimodal methods by a large margin. Furthermore, the probing experiments validate the effectiveness of our method in enhancing translation under the low-resource and massively multilingual scenario.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
