RAGSmith：一种跨数据集寻找检索增强生成方法最优组合的框架
RAGSmith: A Framework for Finding the Optimal Composition of Retrieval-Augmented Generation Methods Across Datasets
AI总结：
提出RAGSmith框架，将RAG设计视为端到端架构搜索，通过遗传搜索联合优化检索与生成指标，在6个领域上平均比朴素基线提升3.8%，并发现向量检索加生成后反思的稳健骨干结构。
摘要：
检索增强生成（RAG）的质量取决于检索、排序、增强、提示和生成过程中的许多相互作用的选择，因此孤立地优化模块非常脆弱。我们提出了RAGSmith，这是一个模块化框架，将RAG设计视为在9个技术族和46,080种可行流水线配置上的端到端架构搜索。遗传搜索优化一个标量目标，该目标联合聚合检索指标（recall@k、mAP、nDCG、MRR）和生成指标（LLM-Judge和语义相似度）。我们在6个Wikipedia衍生领域（数学、法律、金融、医学、国防工业、计算机科学）上进行评估，每个领域包含100个问题，涵盖事实、解释和长答案类型。RAGSmith找到的配置平均比朴素RAG基线高出+3.8%（各领域范围为+1.2%至+6.9%），其中检索增益高达+12.5%，生成增益高达+7.5%。搜索通常探索约0.2%的空间（约100个候选者），并发现一个稳健的骨干结构——向量检索加生成后反思/修订——由扩展、重排序、增强和提示重排序中的领域相关选择进行增强；段落压缩从未被选中。改进幅度与问题类型相关，在事实/长答案混合集中比以解释为主的集合增益更大。这些结果为组装有效的RAG系统提供了实用的、领域感知的指导，并证明了进化搜索在全流水线优化中的实用性。
英文摘要：
Retrieval-Augmented Generation (RAG) quality depends on many interacting choices across retrieval, ranking, augmentation, prompting, and generation, so optimizing modules in isolation is brittle. We introduce RAGSmith, a modular framework that treats RAG design as an end-to-end architecture search over nine technique families and 46{,}080 feasible pipeline configurations. A genetic search optimizes a scalar objective that jointly aggregates retrieval metrics (recall@k, mAP, nDCG, MRR) and generation metrics (LLM-Judge and semantic similarity). We evaluate on six Wikipedia-derived domains (Mathematics, Law, Finance, Medicine, Defense Industry, Computer Science), each with 100 questions spanning factual, interpretation, and long-answer types. RAGSmith finds configurations that consistently outperform naive RAG baseline by +3.8\% on average (range +1.2\% to +6.9\% across domains), with gains up to +12.5\% in retrieval and +7.5\% in generation. The search typically explores $\approx 0.2\%$ of the space ($\sim 100$ candidates) and discovers a robust backbone -- vector retrieval plus post-generation reflection/revision -- augmented by domain-dependent choices in expansion, reranking, augmentation, and prompt reordering; passage compression is never selected. Improvement magnitude correlates with question type, with larger gains on factual/long-answer mixes than interpretation-heavy sets. These results provide practical, domain-aware guidance for assembling effective RAG systems and demonstrate the utility of evolutionary search for full-pipeline optimization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
