M3DR：迈向通用多语言多模态文档检索
M3DR: Towards Universal Multilingual Multimodal Document Retrieval
AI总结：
M3DR提出了一种通用多语言多模态文档检索框架，通过对比训练实现跨语言和跨模态对齐，显著提升了多语言场景下的检索性能。
摘要：
多模态文档检索系统在对齐视觉和文本内容以实现语义搜索方面已取得显著进展。然而，大多数现有方法仍然以英语为中心，限制了其在多语言环境中的有效性。在本工作中，我们提出了M3DR（多语言多模态文档检索），一种框架，旨在跨语言弥合这一差距，使系统能够适用于多样化的语言和文化背景。M3DR利用合成多语言文档数据，并在不同的视觉-语言架构和模型大小之间进行泛化，实现稳健的跨语言和跨模态对齐。通过对比训练，我们的模型学习了文本和文档图像的统一表示，这些表示能够有效跨语言转移。我们在22种类型学上多样的语言上验证了这一能力，展示了在语言和书写系统变化中的稳定性能和适应性。我们进一步引入了一个全面的基准测试，捕捉现实中的多语言场景，评估模型在单语言、多语言和混合语言设置下的表现。M3DR在单密集向量和ColBERT式token级多向量检索范式之间进行泛化。我们的模型NetraEmbed和ColNetraEmbed在跨语言检索上实现了约150%的相对提升，达到了最先进的性能。
英文摘要：
Multimodal document retrieval systems have shown strong progress in aligning visual and textual content for semantic search. However, most existing approaches remain heavily English-centric, limiting their effectiveness in multilingual contexts. In this work, we present M3DR (Multilingual Multimodal Document Retrieval), a framework designed to bridge this gap across languages, enabling applicability across diverse linguistic and cultural contexts. M3DR leverages synthetic multilingual document data and generalizes across different vision-language architectures and model sizes, enabling robust cross-lingual and cross-modal alignment. Using contrastive training, our models learn unified representations for text and document images that transfer effectively across languages. We validate this capability on 22 typologically diverse languages, demonstrating consistent performance and adaptability across linguistic and script variations. We further introduce a comprehensive benchmark that captures real-world multilingual scenarios, evaluating models under monolingual, multilingual, and mixed-language settings. M3DR generalizes across both single dense vector and ColBERT-style token-level multi-vector retrieval paradigms. Our models, NetraEmbed and ColNetraEmbed achieve state-of-the-art performance with ~150% relative improvements on cross-lingual retrieval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
