ICU：通过将任务划分为图像描述生成和语言理解来攻克视觉-语言建模中的语言障碍
ICU: Conquering Language Barriers in Vision-and-Language Modeling by Dividing the Tasks into Image Captioning and Language Understanding
AI总结：
提出ICU方法，将视觉-语言任务分解为英文图像描述与多语言理解两阶段，利用多语言文本数据优势，在IGLUE基准9种语言上取得5项最先进结果。
摘要：
大多数多语言视觉-语言（V&L）研究旨在单一模型内实现多语言和多模态能力。然而，图像的多语言描述稀缺阻碍了其发展。为克服这一障碍，我们提出ICU（图像描述理解），它将V&L任务分为两个阶段：V&L模型用英语生成图像描述，多语言语言模型（mLM）随后将该描述作为替代文本执行跨语言语言理解。多语言处理的负担从V&L模型转移至mLM。由于多语言文本数据相对更丰富且质量更高，ICU有助于攻克V&L模型的语言障碍。在IGLUE基准中覆盖9种语言的两项任务实验中，我们表明ICU在5种语言上取得了新的最先进结果，并在其余语言上取得了相当的结果。
英文摘要：
Most multilingual vision-and-language (V&L) research aims to accomplish multilingual and multimodal capabilities within one model. However, the scarcity of multilingual captions for images has hindered the development. To overcome this obstacle, we propose ICU, Image Caption Understanding, which divides a V&L task into two stages: a V&L model performs image captioning in English, and a multilingual language model (mLM), in turn, takes the caption as the alt text and performs cross-lingual language understanding. The burden of multilingual processing is lifted off V&L model and placed on mLM. Since the multilingual text data is relatively of higher abundance and quality, ICU can facilitate the conquering of language barriers for V&L models. In experiments on two tasks across 9 languages in the IGLUE benchmark, we show that ICU can achieve new state-of-the-art results for five languages, and comparable results for the rest.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
