量化多模态多语言检索训练中翻译与母语感知之间的差距
Quantifying the Gaps Between Translation and Native Perception in Training for Multimodal, Multilingual Retrieval
AI总结：
针对多语言视觉语言模型缺乏跨语言文化感知差异考量的问题，本研究通过多模态多语言检索案例量化训练数据的性能差距，提出描述增强策略并实现1.3的平均召回率提升，但差距仍存，为后续研究指明方向。
摘要：
目前缺乏能恰当反映不同语言和文化下图片描述所体现的感知差异的多语言视觉语言模型。本研究通过多模态多语言检索案例研究，量化了现有模型灵活性的不足。我们通过实验证明，在基于德语母语感知生成的描述上训练，与在从英语机器翻译或人工翻译为德语的描述上训练之间存在性能差距。为弥补这些差距，我们进一步提出并评估了描述增强策略。尽管我们实现了平均召回率提升（+1.3），但差距仍然存在，这为学界留下了未来研究的开放方向。
英文摘要：
There is a scarcity of multilingual vision-language models that properly account for the perceptual differences that are reflected in image captions across languages and cultures. In this work, through a multimodal, multilingual retrieval case study, we quantify the existing lack of model flexibility. We empirically show performance gaps between training on captions that come from native German perception and captions that have been either machine-translated or human-translated from English into German. To address these gaps, we further propose and evaluate caption augmentation strategies. While we achieve mean recall improvements (+1.3), gaps still remain, indicating an open area of future work for the community.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
