arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.26499cs.AI
收藏

AIRA_2:克服人工智能研究代理中的瓶颈

AIRA_2: Overcoming Bottlenecks in AI Research Agents

FAIR at Meta(Meta FAIR);University College London(伦敦大学学院);University of Oxford(牛津大学)
Karen Hambardzumyan, Nicolas Baldwin, Edan Toledo, Rishi Hazra, Michael Kuchnik, Bassel Al Omari, Thomas Simon Foster, Anton Protopopov, Jean-Christophe Gagnon-… 展开作者

Karen Hambardzumyan, Nicolas Baldwin, Edan Toledo, Rishi Hazra, Michael Kuchnik, Bassel Al Omari, Thomas Simon Foster, Anton Protopopov, Jean-Christophe Gagnon-Audet, Ishita Mediratta, Kelvin Niu, Michael Shvartsman, Alisia Lupidi, Alexis Audran-Reiss, Parth Pathak, Tatiana Shavrina, Despoina Magka, Hela Momand, Derek Dunfield, Nicola Cancedda, Pontus Stenetorp, Carole-Jean Wu, Jakob Nicolaus Foerster, Yoram Bachrach, Martin Josifoski

AI总结:

AIRA_2通过异步多GPU工作池、隐藏一致性评估协议和ReAct代理解决AI研究代理的三个性能瓶颈,提升搜索性能。

摘要:

现有研究指出了人工智能研究代理的三个结构性性能瓶颈:(1)同步单GPU执行限制样本吞吐量,限制搜索收益;(2)泛化差距,基于验证的选择导致过拟合,搜索时间延长时性能下降;(3)固定单轮LLM操作符的有限能力限制搜索性能。我们引入AIRA$_2$,通过三个架构选择解决这些瓶颈:异步多GPU工作池线性提高实验吞吐量;隐藏一致性评估协议提供可靠评估信号;ReAct代理动态限定其行为并交互调试。在MLE-bench-30上,AIRA$^{\dagger}_{2}$在24小时内达到81.5%的百分位排名,在72小时内达到83.1%,优于最强基线72.7%。在AIRS-Bench上,AIRA_2在20个多样化研究任务中超越人类当前最佳水平。消融实验确认每个架构组件都是必要的,性能遵循可预测的扩展定律,跨LLM基础架构转移,且先前工作报告的'过拟合'是由评估噪声而非真实数据记忆驱动。

英文摘要:

Existing research has identified three structural performance bottlenecks in AI research agents: (1) synchronous single-GPU execution constrains sample throughput, limiting the benefit of search; (2) a generalization gap where validation-based selection causes overfitting and performance to degrade over extended search horizons; and (3) the limited capability of fixed, single-turn LLM operators imposes a ceiling on search performance. We introduce AIRA$_2$, which addresses these bottlenecks through three architectural choices: an asynchronous multi-GPU worker pool that increases experiment throughput linearly; a Hidden Consistent Evaluation protocol that delivers a reliable evaluation signal; and ReAct agents that dynamically scope their actions and debug interactively. On MLE-bench-30, AIRA$^{\dagger}_{2}$ achieves a mean Percentile Rank of 81.5% at 24 hours and 83.1% at 72 hours, outperforming the strongest baseline, which achieves 72.7%. On AIRS-Bench, AIRA$_2$ exceeds human state-of-the-art on 6 out of 20 diverse research tasks. Ablations confirm that each architectural component is necessary, that performance follows a predictable scaling law that transfers across LLM backbones, and that the "overfitting" reported in prior work was driven by evaluation noise rather than true data memorization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑