mFollowIR：一个用于检索中指令遵循的多语言基准
mFollowIR: a Multilingual Benchmark for Instruction Following in Retrieval
AI总结：
本文提出多语言检索基准 mFollowIR，通过修改 TREC NeuCLIR 叙述测试模型跨三种语言的指令遵循能力，发现基于英语指令的检索器跨语言表现优异，但多语言场景下性能显著下降。
摘要：
检索系统通常专注于简短且未充分说明的网络风格查询。然而，语言模型的进步促进了能够理解具有多样化意图的复杂查询的检索模型的初步兴起。然而，这些努力完全集中在英语上；因此，我们尚不清楚它们在跨语言环境下的表现。我们引入了 mFollowIR，一个用于衡量检索模型中指令遵循能力的多语言基准。mFollowIR 建立在 TREC NeuCLIR 的叙述（或指令）之上，涵盖三种不同的语言（俄语、中文、波斯语），同时向检索模型提供查询和指令。我们对这些叙述进行了微小的修改，以隔离并测试检索模型遵循这些细微变化的能力。我们展示了多语言（XX-XX）和跨语言（En-XX）的性能结果。我们观察到，使用指令训练的基于英语的检索器在跨语言性能上表现强劲，但发现在多语言设置中性能显著下降，这表明在为基于指令的多语言检索器开发数据方面仍需更多工作。
英文摘要：
Retrieval systems generally focus on web-style queries that are short and underspecified. However, advances in language models have facilitated the nascent rise of retrieval models that can understand more complex queries with diverse intents. However, these efforts have focused exclusively on English; therefore, we do not yet understand how they work across languages. We introduce mFollowIR, a multilingual benchmark for measuring instruction-following ability in retrieval models. mFollowIR builds upon the TREC NeuCLIR narratives (or instructions) that span three diverse languages (Russian, Chinese, Persian) giving both query and instruction to the retrieval models. We make small changes to the narratives and isolate how well retrieval models can follow these nuanced changes. We present results for both multilingual (XX-XX) and cross-lingual (En-XX) performance. We see strong cross-lingual performance with English-based retrievers that trained using instructions, but find a notable drop in performance in the multilingual setting, indicating that more work is needed in developing data for instruction-based multilingual retrievers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
