更少微调，更好检索：通过合成数据和模型合并重新思考LLM适应于生物医学检索器
Modular Expert Merging for Biomedical Retrieval
AI总结：
本文提出STM框架，通过合成数据和模型合并提升生物医学检索性能，实验显示在任务特定专家上提升23.5%并优于基线模型。
摘要：
检索增强生成（RAG）已成为使大型语言模型（LLM）接地的骨干，提高了知识更新并减少了幻觉。最近，基于LLM的检索模型在RAG应用中表现出最先进的性能。然而，有几个技术方面仍然未被探索，即如何将通用目的LLM适应为有效的领域特定检索器，特别是在生物医学等专业领域。我们提出了Synthesize-Train-Merge（STM），一个模块化框架，通过合成硬负例、检索提示优化和模型合并来增强解码器-only LLMs。在MTEB基准测试中12个医学和通用任务的子集上的实验显示，STM将任务特定专家提高了最高23.5%（平均7.5%），并产生优于单个专家和强大基线的合并模型，而无需广泛的预训练。我们的结果展示了一条可扩展、高效的路径，将通用LLM转化为高性能、领域专用的检索器，在保持通用领域能力的同时在专门任务上表现出色。
英文摘要：
Adapting general-purpose LLMs into domain-specialized dense retrievers typically requires large-scale training on mixed-domain data. We show that merging independently trained domain-specialized experts consistently exceeds this approach across four decoder-only LLM families (0.6B-7B), four merging methods, and twelve medical and general retrieval tasks from MTEB, suggesting that parameter-space composition captures complementary domain strengths that large-scale mixed-domain training averages out. To further maximize expert quality, we introduce Synthesize-Train-Merge (STM), a modular framework that synthesizes hard negatives with a top-tier LLM and fine-tunes domain-specialized experts via LoRA before merging them, without continual pre-training. Synthesized hard negatives yield the largest gains for smaller models, and STM achieves strong performance on biomedical retrieval tasks while maintaining competitive general-domain results across all four backbone families.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
