利用RAG进行微调以提升大语言模型的新技能学习能力
Fine-tuning with RAG for Improving LLM Learning of New Skills
AI总结：
针对LLM智能体多步任务易失败、RAG依赖外部库且开销高的问题，提出将检索知识蒸馏为模型内在能力的微调流程，在两个基准上性能优于基线，且泛化性好、token开销更低。
摘要：
部署用于多步骤任务的大语言模型（LLM）智能体常会出现可预测的失败：尝试执行前提条件未满足的动作、发出冗余指令，或是对环境约束处理不当。虽然检索增强生成（RAG）可通过提供运行时指导提升性能，但它需要维护外部知识库，且每次部署都会增加计算开销。我们提出一种简单流程，通过蒸馏将推理时的检索转化为习得的能力。我们的方法包括：（1）从智能体失败案例中提取紧凑、可复用的提示；（2）在回合开始时通过单次检索，利用这些提示生成更优的教师轨迹；（3）在移除提示字符串的这些轨迹上训练学生模型，迫使模型内化知识而非死记硬背。在两个交互式基准ALFWorld（家务任务）和WebShop（在线购物）上，蒸馏后的学生模型始终优于基线智能体，在ALFWorld上成功率最高达91%（基线为79%），WebShop得分提升至72（基线为61），同时根据环境不同，token使用量比检索增强的教师模型少10%-60%。该方法在不同模型规模（7B/14B参数）和智能体架构（ReAct/StateAct）下均具备泛化性，证明检索带来的收益可通过针对性微调有效内化，无需永久的运行时依赖。
英文摘要：
Large language model (LLM) agents deployed for multi-step tasks frequently fail in predictable ways: attempting actions with unmet preconditions, issuing redundant commands, or mishandling environment constraints. While retrieval-augmented generation (RAG) can improve performance by providing runtime guidance, it requires maintaining external knowledge databases and adds computational overhead at every deployment. We propose a simple pipeline that converts inference-time retrieval into learned competence through distillation. Our approach: (1) extracts compact, reusable hints from agent failures, (2) uses these hints to generate improved teacher trajectories via one-shot retrieval at episode start, and (3) trains student models on these trajectories with hint strings removed, forcing internalization rather than memorization. Across two interactive benchmarks, ALFWorld (household tasks) and WebShop (online shopping), distilled students consistently outperform baseline agents, achieving up to 91% success on ALFWorld (vs. 79% for baselines) and improving WebShop scores to 72 (vs. 61 for baselines), while using 10-60% fewer tokens than retrieval-augmented teachers depending on the environment. The approach generalizes across model scales (7B/14B parameters) and agent architectures (ReAct/StateAct), demonstrating that retrieval benefits can be effectively internalized through targeted fine-tuning without permanent runtime dependencies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
