合成混合训练:在RAG之上扩展参数化知识获取
Synthetic Mixed Training: Scaling Parametric Knowledge Acquisition Beyond RAG
AI总结:
本文提出合成混合训练方法,结合合成问答和文档,通过互补训练信号提升模型性能,在QuaLITY基准上实现4.4%的相对提升。
摘要:
合成数据增强帮助语言模型在数据受限领域学习新知识。然而,简单地通过训练更多合成token或使用更强的生成器扩展现有合成数据方法,其性能在超越RAG后趋于平缓。为突破RAG上限,我们引入合成混合训练,结合合成问答和文档。该方法利用其互补的训练信号,并随着合成数据量和生成器强度的增加,实现对数线性提升。这使模型在QuaLITY长文档阅读理解基准上超越RAG 2.6%。此外,我们引入聚焦重写,一种简单的合成文档生成技术,通过显式将文档生成条件于特定问题,提高合成文档的多样性,并产生更陡峭的对数线性缩放曲线。在QuaLITY上,我们的最终配方训练了一个Llama 8B模型,相对超越RAG 4.4%。在多个模型和基准(QuaLITY、LongHealth、FinanceBench)上,我们的训练使模型在六个设置中的五个超越RAG,相对提升2.6%,并与RAG结合时达到9.1%的提升。
英文摘要:
Synthetic data augmentation helps language models learn new knowledge in data-constrained domains. However, naively scaling existing synthetic data methods by training on more synthetic tokens or using stronger generators yields diminishing returns below the performance of RAG. To break the RAG ceiling, we introduce Synthetic Mixed Training, which combines synthetic QAs and synthetic documents. This leverages their complementary training signals, and enables log-linear improvements as both synthetic data volume and generator strength increase. This allows the model to outperform RAG by a 2.6% relative gain on QuaLITY, a long-document reading comprehension benchmark. In addition, we introduce Focal Rewriting, a simple technique for synthetic document generation that explicitly conditions document generation on specific questions, improving the diversity of synthetic documents and yielding a steeper log-linear scaling curve. On QuaLITY, our final recipe trains a Llama 8B model that outperforms RAG by 4.4% relatively. Across models and benchmarks (QuaLITY, LongHealth, FinanceBench), our training enables models to beat RAG in five of six settings, outperforms by 2.6%, and achieves a 9.1% gain when combined with RAG.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。