arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.28409cs.AI
收藏

基于离线强化学习的代码生成LLM高效后训练

Efficient Post-training of LLMs for Code Generation With Offline Reinforcement Learning

Hessian Center for Artificial Intelligence(海德堡人工智能中心);National Research Center for Applied Cybersecurity ATHENE(应用网络安全国家研究中心ATHENE)

Mingze Wu, Abhinav Anand, Shweta Verma, Mira Mezini

AI总结:

本文探索使用离线强化学习利用现有代码数据集对代码生成LLM进行后训练,实验表明该方法能有效提升模型性能,尤其适用于小模型和复杂编码问题。

摘要:

使用在线强化学习(RL)进行后训练是LLM(包括代码生成模型)的重要训练步骤。然而,用于代码生成的在线RL涉及LLM推理和生成输出的验证,这可能耗费大量时间和资源。在本文中,我们通过利用现有代码数据集,探索将离线RL应用于代码生成模型。我们的实验表明,离线RL是提升LLM性能的有效训练策略。我们证明,离线RL对于小型LLM和具有挑战性的编码问题尤其有益。

英文摘要:

Post-training using online reinforcement learning (RL) is an important training step for LLMs, including code-generating models. However, online RL for code generation involves LLM inference and verification of the generated output, which can take considerable time and resources. In this paper, we explore the application of offline RL to code-generating models by leveraging existing code datasets. Our experiments demonstrate that offline RL is an effective training strategy for improving LLM performance. We show that offline RL can be especially beneficial for small LLMs and challenging coding problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑