arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.02556cs.LGcs.AI
收藏

超越经验检索:学习生成优化的结构化经验以冻结LLM

Beyond Experience Retrieval: Learning to Generate Utility-Optimized Structured Experience for Frozen LLMs

Department of Computer Science and Technology, Tsinghua University, Beijing, China(清华大学计算机科学与技术系,北京,中国);Quancheng Laboratory(千晨实验室)

Xuancheng Li, Haitao Li, Yujia Zhou, Yiqun Liu, Qingyao Ai

AI总结:

本文提出SEAM模块,通过单次前向传递生成实例定制的经验条目,优化冻结LLM的执行效率,实验显示在数学推理基准上准确率提升显著。

摘要:

大型语言模型(LLMs)大多静态且常常重复推理或重复错误。先前的经验重用通常依赖于外部检索,这种基于相似性的方法会引入噪声并增加延迟。我们引入SEAM(结构化经验适配模块),这是一种轻量级、执行器特定的插件,它在参数中存储经验,并在单次前向传递中生成实例定制的经验条目,以指导冻结的LLM执行器。SEAM通过执行器的回放和GRPO进行训练以实现实用性,并在部署后通过在记录的成功轨迹上进行有监督的微调进一步改进。在数学推理基准上的实验显示,执行器在低开销下实现了持续的准确性提升。广泛的消融分析和研究进一步阐明了SEAM有效性和鲁棒性的机制。

英文摘要:

Large language models (LLMs) are largely static and often redo reasoning or repeat mistakes. Prior experience reuse typically relies on external retrieval, which is similarity-based, can introduce noise, and adds latency. We introduce SEAM (Structured Experience Adapter Module), a lightweight, executor-specific plug-in that stores experience in its parameters and generates a structured, instance-tailored experience entry in a single forward pass to guide a frozen LLM executor. SEAM is trained for utility via executor rollouts and GRPO while keeping the executor frozen, and it can be further improved after deployment with supervised fine-tuning on logged successful trajectories. Experiments on mathematical reasoning benchmarks show consistent accuracy gains across executors with low overhead. Extensive ablations and analyses further elucidate the mechanisms underlying SEAM's effectiveness and robustness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑