RouterRetriever：在专家嵌入模型混合体上进行路由
RouterRetriever: Routing over a Mixture of Expert Embedding Models
AI总结：
RouterRetriever通过路由机制在混合领域专家嵌入模型中选择最合适专家，轻量且可扩展，在BEIR基准上超越单一模型和多任务模型，展示了路由替代通用嵌入模型的有效性。
摘要：
信息检索方法通常依赖于在大型通用领域数据集（如MSMARCO）上训练的单一嵌入模型。虽然这种方法可以产生具有合理整体性能的检索器，但在各自领域进行测试时，它们往往不如在领域特定数据上训练的模型。先前在信息检索领域的工作通过多任务训练来解决这个问题，但在语言模型生成研究中，尽管这种思想很流行，在混合领域特定专家检索器上进行路由的想法仍未得到探索。在这项工作中，我们引入了RouterRetriever，一种利用混合领域特定专家的检索模型，通过路由机制为每个查询选择最合适的专家。RouterRetriever是轻量级的，允许轻松添加或移除专家，无需额外训练。在BEIR基准上的评估表明，RouterRetriever优于在MSMARCO上训练的模型（+2.1绝对nDCG@10）和多任务模型（+3.2）。这是通过采用我们的路由机制实现的，该机制超越了语言建模中常用的其他路由技术（平均+1.8）。此外，这种优势很好地泛化到其他数据集，即使在数据集上没有特定专家的情况下也是如此。RouterRetriever是第一个证明在混合领域特定专家嵌入模型上进行路由作为单一通用嵌入模型替代方案优势的工作，特别是在从多样化的专业领域进行检索时。
英文摘要：
Information retrieval methods often rely on a single embedding model trained on large, general-domain datasets like MSMARCO. While this approach can produce a retriever with reasonable overall performance, they often underperform models trained on domain-specific data when testing on their respective domains. Prior work in information retrieval has tackled this through multi-task training, but the idea of routing over a mixture of domain-specific expert retrievers remains unexplored despite the popularity of such ideas in language model generation research. In this work, we introduce RouterRetriever, a retrieval model that leverages a mixture of domain-specific experts by using a routing mechanism to select the most appropriate expert for each query. RouterRetriever is lightweight and allows easy addition or removal of experts without additional training. Evaluation on the BEIR benchmark demonstrates that RouterRetriever outperforms both models trained on MSMARCO (+2.1 absolute nDCG@10) and multi-task models (+3.2). This is achieved by employing our routing mechanism, which surpasses other routing techniques (+1.8 on average) commonly used in language modeling. Furthermore, the benefit generalizes well to other datasets, even in the absence of a specific expert on the dataset. RouterRetriever is the first work to demonstrate the advantages of routing over a mixture of domain-specific expert embedding models as an alternative to a single, general-purpose embedding model, especially when retrieving from diverse, specialized domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
