arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.17547cs.AIcs.CL
收藏

KLong:训练用于超长 horizon 任务的 LLM 代理

KLong: Training LLM Agent for Extremely Long-horizon Tasks

Yue Liu

AI总结:

KLong 通过轨迹分割 SFT 和渐进式 RL 训练,解决超长 horizon 任务,实现 106B 模型在 PaperBench 上超越 Kimi K2 Thinking 11.28%。

摘要:

本文介绍了KLong,一种开源的LLM代理,旨在解决超长horizon任务。其原理是首先通过轨迹分割SFT冷启动模型,然后通过渐进式RL训练进行扩展。具体而言,我们首先使用全面的SFT配方激活基础模型的基本代理能力。然后,我们引入Research-Factory,一个自动化管道,通过收集研究论文和构建评估标准来生成高质量的训练数据。利用该管道,我们从Claude 4.5 Sonnet(Thinking)中构建了数千条超长horizon轨迹。为了训练这些极长的轨迹,我们提出了一种新的轨迹分割SFT,该方法保留早期上下文,逐步截断后期上下文,并保持子轨迹之间的重叠。此外,为了进一步提高超长horizon任务解决能力,我们提出了一种新的渐进式RL,将训练分为多个阶段,逐步延长超时时间。实验表明KLong的优越性和泛化能力,如图1所示。值得注意的是,我们的KLong(106B)在PaperBench上超越Kimi K2 Thinking(1T)11.28%,且性能提升泛化到其他编码基准如SWE-bench Verified和MLE-bench。

英文摘要:

This paper introduces KLong, an open-source LLM agent trained to solve extremely long-horizon tasks. The principle is to first cold-start the model via trajectory-splitting SFT, then scale it via progressive RL training. Specifically, we first activate basic agentic abilities of a base model with a comprehensive SFT recipe. Then, we introduce Research-Factory, an automated pipeline that generates high-quality training data by collecting research papers and constructing evaluation rubrics. Using this pipeline, we build thousands of long-horizon trajectories distilled from Claude 4.5 Sonnet (Thinking). To train with these extremely long trajectories, we propose a new trajectory-splitting SFT, which preserves early context, progressively truncates later context, and maintains overlap between sub-trajectories. In addition, to further improve long-horizon task-solving capability, we propose a novel progressive RL, which schedules training into multiple stages with progressively extended timeouts. Experiments demonstrate the superiority and generalization of KLong, as shown in Figure 1. Notably, our proposed KLong (106B) surpasses Kimi K2 Thinking (1T) by 11.28% on PaperBench, and the performance improvement generalizes to other coding benchmarks like SWE-bench Verified and MLE-bench.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑