发表机构
KAIST(韩国科学技术院)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
提出DroidTool框架,通过智能体工作流自生成工具操作,增强Android GUI智能体,在三个基准上平均性能提升4.47个百分点,交互减少20.05%。
AI 中文摘要
采用结合GUI操作与工具操作(例如通过API访问应用数据)的混合动作空间的Android智能体仍 largely 未被充分探索,主要原因是创建工具需要过多的工作量。为弥补这一空白,我们提出了DroidTool,一个通过自生成工具来增强智能体的框架,这些工具以操作应用状态(如数据库)的Python函数形式实现。为了以最少的人力创建工具,DroidTool采用了一个包含提案、实现、测试生成与执行以及修复等阶段的智能体工作流。值得注意的是,在验证所创建工具时,它构建了跨相关工具的关系测试,以便自然准备适当的测试前提条件并提高测试覆盖率,而不是单独测试每个工具。在代表性基准测试AndroidWorld、B-MoCA和MobileSafetyBench上,使用生成工具增强的GUI智能体平均实现了约4.47个百分点的性能提升,同时交互次数减少了约20.05%,相比仅使用GUI的智能体。
英文摘要
Android agents using a hybrid action space that combines GUI actions and tool actions (e.g., accessing application data via APIs) remain largely underexplored, mainly due to the excessive effort required to create tools. To address this gap, we introduce DroidTool, a framework for augmenting the agents with self-generated tools, which are realized as Python functions operating on application states (e.g., a database). To create tools with minimal human labor, DroidTool employs an agentic workflow featuring stages: proposal, implementation, test generation and execution, and repair. Notably, when testing the created tools for verification, it constructs relational tests across relevant tools for natural preparation of appropriate test preconditions and improved test coverage, rather than testing each tool separately. The GUI agents augmented with the generated tools achieved approximately 4.47%p higher performance with approximately 20.05% fewer interactions than the GUI-only agents, averaged across representative benchmarks: AndroidWorld, B-MoCA, and MobileSafetyBench.