arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.28601cs.AI
收藏

演化智能体界面

Evolving Agent Interfaces

University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校);Cornell University(康奈尔大学);Language Technologies Institute, Carnegie Mellon University(卡内基梅隆大学语言技术研究所);Simular

Saaket Agashe, Yuzhe Yang, Gonzalo Gonzalez-Pumariega, Bhuvan Chandra Koduru, Chih-Lun Li, Jiachen Yang, Ang Li, Xin Eric Wang

AI总结:

本文提出将界面设计作为智能体能力,引入Agent S4,通过元智能体演化界面存储库,在OSWorld V2和ScienceBoard上大幅提升计算机使用智能体性能,且成本远低于GPT-5.5等模型。

摘要:

计算机使用智能体受限于人类为其提供的界面。智能体可能会逐个单元格地在电子表格中输入内容,而实际上只需一行代码即可完成;或者通过Shell打印出一个文件的全部2000行内容来查找某个函数。界面以及何时使用何种界面的知识均为手动构建,无法跟上新应用、工作流和模型的发展速度。本文提出,界面设计本身应成为智能体的一项能力。我们引入Agent S4,这是一款自主计算机使用智能体,可基于任务经验构建演化智能体界面。一个元智能体在合成训练任务上检查任务智能体的轨迹(无需成功标签),并在一个存储库中构建和修改界面与使用知识。随后,基于种群的演化通过元智能体从未见过的验证任务上的表现来选择哪些存储库需要进一步开发,偏好那些能泛化到已见任务之外的变更。在OSWorld V2上,搭载Qwen3.8-27B及演化后的界面存储库的Agent S4,在开源计算机使用智能体中创下新的最优性能,达到21.3%的二元得分,是使用固定界面的同模型性能的3.8倍。该性能超过GPT-5.5(13.0%)和Claude Opus 4.8(20.6%),而成本仅为后者的约4.3%和1.5%。在ScienceBoard上,使用相同模型时,演化后的界面将成功率从41.4%提升至58.6%。这些结果表明,可从经验中演化出有用的界面,使自主界面设计成为与模型及工具改进互补的提升来源。我们在该https URL开源了代码。

英文摘要:

Computer-use agents are limited by the interfaces humans give them. An agent may type a spreadsheet cell by cell when a single line of code would do, or print all two thousand lines of a file through a shell to find one function. Both the interfaces and the knowledge of when to use each are built by hand, which cannot keep pace with new applications, workflows, and models. We argue that interface design should itself be an agent capability. We introduce Agent S4, an autonomous computer-use scientist that builds Evolving Agent Interfaces from task experience. A meta-agent examines task agents' trajectories on synthetic training tasks, without success labels, and constructs and revises interfaces and usage knowledge in a repository. Population-based evolution then selects which repositories to develop further by their performance on validation tasks the meta-agent never sees, favoring changes that generalize beyond the seen tasks. On OSWorld V2, Agent S4, with Qwen3.8-27B and the evolved repository of interfaces, sets a new state of the art in open-source computer-use agents, reaching 21.3% binary score, 3.8 times the same model with fixed interfaces. This is above both GPT-5.5 (13.0%) and Claude Opus 4.8 (20.6%), at about 4.3% and 1.5% of their respective costs. On ScienceBoard, the evolved interfaces improve the success rate from 41.4% to 58.6% with the same model. These results suggest that useful interfaces can be evolved from experience, making autonomous interface design a source of improvement complementary to advances in models and harnesses. We open-source our code at https://github.com/simular-ai/agent-s.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑