回归基础：提升稠密编码器域外检索性能的简单配方
Back to Basics: A Simple Recipe for Improving Out-of-Domain Retrieval in Dense Encoders
AI总结：
本工作提出在MSMARCO上使用LoRA等参数高效方法并采用批内负样本的简单训练配方，经BEIR基准验证，能有效提升稠密编码器的域外泛化性能，且与其他资源密集型策略互补。
摘要：
当前主流的研究实践通常依赖于在现有大型数据集（如MSMARCO）上训练稠密检索器，然后尝试各种方法来提升其在未见领域上的零样本泛化能力。尽管先前的工作通过数据增强、架构修改、增大模型规模甚至进一步预训练基础模型等资源密集型步骤来应对这一挑战，但相对较少的研究关注训练过程本身是否可以被改进，以使最终模型获得更好的泛化能力。在本工作中，我们推荐一个训练稠密编码器的简单配方：在MSMARCO上使用参数高效方法（如LoRA）进行训练，并选择使用批内负样本，除非有构造良好的难负样本可用。我们使用BEIR基准验证了这些建议，发现结果在稠密编码器选择和基础模型规模上保持一致，并且与架构修改或额外预训练等其他资源密集型的域外泛化策略互补。我们希望这项围绕各种训练技术的全面且公正的研究，能够为开发一个即使仅在单一数据集上训练也能有效泛化的稠密检索模型提供实用见解。
英文摘要：
Prevailing research practice today often relies on training dense retrievers on existing large datasets such as MSMARCO and then experimenting with ways to improve zero-shot generalization capabilities to unseen domains. While prior work has tackled this challenge through resource-intensive steps such as data augmentation, architectural modifications, increasing model size, or even further base model pretraining, comparatively little investigation has examined whether the training procedures themselves can be improved to yield better generalization capabilities in the resulting models. In this work, we recommend a simple recipe for training dense encoders: Train on MSMARCO with parameter-efficient methods, such as LoRA, and opt for using in-batch negatives unless given well-constructed hard negatives. We validate these recommendations using the BEIR benchmark and find results are persistent across choice of dense encoder and base model size and are complementary to other resource-intensive strategies for out-of-domain generalization such as architectural modifications or additional pretraining. We hope that this thorough and impartial study around various training techniques, which augments other resource-intensive methods, offers practical insights for developing a dense retrieval model that effectively generalizes, even when trained on a single dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
