arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.06140cs.CL
收藏

CurateEvo:用于智能体训练后的数据管理进化

CurateEvo: Data-Curation Evolving for Agentic Post-Training

Dingzirui Wang, Xuanliang Zhang, Keyan Xu, Qingfu Zhu, Wanxiang Che

AI总结:

研究针对大语言模型智能体训练后决策问题,提出CurateEvo框架,将数据管理策略表示为代码并迭代重写。通过诊断故障模式增强、过滤数据提高有效性,在成本感知下修剪训练轮次提高效率,实验证明其性能优于现有方法且降低管理开销。

摘要:

大语言模型智能体需要训练后方法来根据环境反馈改善长期决策。现有智能体训练后流程常将数据管理视为固定预处理步骤,主要关注数据增强而忽视过滤、细化及适应下游故障。我们提出CurateEvo,一种用于智能体训练后数据管理的故障驱动动态进化框架。它将管理策略表示为可执行代码,利用验证集的失败轨迹迭代重写。每个时期,进化策略将原始语料库转化为监督微调数据、强化学习数据和推理时记忆库。先通过诊断重复故障模式并相应增强、过滤或细化数据提高有效性,再在成本感知目标下修剪冗余或低效用训练轮次提高效率。在多个数据集上实验表明CurateEvo优于现有方法,还兼容不同训练后方法并大幅降低管理开销。

英文摘要:

Large language model (LLM) agents require post-training methods that can improve long-horizon decision making from environment feedback. However, existing agentic post-training pipelines often treat data curation as a fixed preprocessing step, focusing mainly on data augmentation while neglecting filtering, refinement, and adaptation to downstream failures. We propose CurateEvo, a failure-driven dynamic evolution framework for agentic post-training data curation. CurateEvo represents the curation strategy as executable code and iteratively rewrites it using failed trajectories from a held-out development set. At each epoch, the evolved strategy transforms a fixed raw corpus into supervised fine-tuning data, reinforcement learning data, and an inference-time memory bank. The evolution process first improves effectiveness by diagnosing recurring failure modes and augmenting, filtering, or refining data accordingly, and then improves efficiency by pruning redundant or low-utility training turns under a cost-aware objective. Experiments on ACEBench-Agent, BFCL-V4, and τ^2-Bench under both labeled and wild-data settings show that CurateEvo consistently outperforms prior curation methods, improving average scores by 3.2 and 2.7 points, respectively. Further analyses demonstrate that CurateEvo is compatible with different post-training recipes and substantially reduces curation overhead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑