arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.07906cs.LGcs.AI
收藏

AceGRPO:自适应课程增强的群体相对策略优化用于自主机器学习工程

AceGRPO: Adaptive Curriculum Enhanced Group Relative Policy Optimization for Autonomous Machine Learning Engineering

School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院)

Yuzhu Cai, Zexi Liu, Xinyu Zhu, Cheng Wang, Yanfeng Wang, Siheng Chen

AI总结:

本文提出AceGRPO,通过动态数据缓冲和可学习潜力函数提升自主机器学习工程的持续迭代优化能力,实验证明其在MLE-Bench-Lite上达到100%有效提交率。

摘要:

自主机器学习工程(MLE)要求智能体在长时间范围内进行持续迭代优化。尽管基于大语言模型的智能体表现出潜力,但当前基于提示的MLE智能体因参数冻结导致行为停滞。虽然强化学习(RL)提供了解决方案,但将其应用于MLE受到执行延迟和数据选择效率的阻碍。针对这些挑战,我们提出AceGRPO,包含两个核心组件:(1)进化数据缓冲器,持续将执行轨迹重新利用为可重用的训练任务;(2)适应性采样,由可学习潜力函数引导,动态优先处理智能体学习前沿的任务以最大化学习效率。利用AceGRPO,训练的Ace-30B模型在MLE-Bench-Lite上实现了100%的有效提交率,接近专有前沿模型的性能,并优于更大的开源基线(如DeepSeek-V3.2),展示了持续迭代优化的鲁棒性。代码可在https://github.com/yuzhu-cai/AceGRPO获取。

英文摘要:

Autonomous Machine Learning Engineering (MLE) requires agents to perform sustained, iterative optimization over long horizons. While recent LLM-based agents show promise, current prompt-based agents for MLE suffer from behavioral stagnation due to frozen parameters. Although Reinforcement Learning (RL) offers a remedy, applying it to MLE is hindered by prohibitive execution latency and inefficient data selection. Recognizing these challenges, we propose AceGRPO with two core components: (1) Evolving Data Buffer that continuously repurposes execution traces into reusable training tasks, and (2) Adaptive Sampling guided by a Learnability Potential function, which dynamically prioritizes tasks at the agent's learning frontier to maximize learning efficiency. Leveraging AceGRPO, our trained Ace-30B model achieves a 100% valid submission rate on MLE-Bench-Lite, approaches the performance of proprietary frontier models, and outperforms larger open-source baselines (e.g., DeepSeek-V3.2), demonstrating robust capability for sustained iterative optimization. Code is available at https://github.com/yuzhu-cai/AceGRPO.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑