PC-Agent:面向PC端复杂任务自动化的分层多智能体协作框架
PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PC
AI总结:
针对PC端GUI智能体交互环境复杂、工作流繁琐的问题,提出分层多智能体框架PC-Agent,搭配主动感知模块与多角色协作机制,在新基准PC-Eval上任务成功率较现有最优方法提升32%。
摘要:
在基于多模态大语言模型(MLLM)的GUI智能体领域,与智能手机场景相比,PC场景不仅交互环境更复杂,还涉及更繁复的应用内与跨应用工作流。为解决这些问题,我们提出了名为PC-Agent的分层智能体框架。具体而言,在感知层面,我们设计了主动感知模块(Active Perception Module, APM),以弥补当前MLLM在截图内容感知方面的能力不足。在决策层面,为更高效地处理复杂用户指令和相互依赖的子任务,我们提出了分层多智能体协作架构,将决策过程分解为“指令-子任务-动作”三个层级。在该架构内,设置了三个智能体(即Manager、Progress和Decision),分别负责指令分解、进度跟踪和逐步决策。此外,还采用了Reflection智能体以实现自下而上的及时错误反馈与调整。我们还推出了包含25条真实复杂指令的新基准PC-Eval。在PC-Eval上的实验结果表明,我们的PC-Agent相比此前的最优方法,任务成功率实现了32%的绝对提升。代码可在https://github.com/X-PLUG/MobileAgent/tree/main/PC-Agent获取。
英文摘要:
In the field of MLLM-based GUI agents, compared to smartphones, the PC scenario not only features a more complex interactive environment, but also involves more intricate intra- and inter-app workflows. To address these issues, we propose a hierarchical agent framework named PC-Agent. Specifically, from the perception perspective, we devise an Active Perception Module (APM) to overcome the inadequate abilities of current MLLMs in perceiving screenshot content. From the decision-making perspective, to handle complex user instructions and interdependent subtasks more effectively, we propose a hierarchical multi-agent collaboration architecture that decomposes decision-making processes into Instruction-Subtask-Action levels. Within this architecture, three agents (i.e., Manager, Progress and Decision) are set up for instruction decomposition, progress tracking and step-by-step decision-making respectively. Additionally, a Reflection agent is adopted to enable timely bottom-up error feedback and adjustment. We also introduce a new benchmark PC-Eval with 25 real-world complex instructions. Empirical results on PC-Eval show that our PC-Agent achieves a 32% absolute improvement of task success rate over previous state-of-the-art methods. The code is available at https://github.com/X-PLUG/MobileAgent/tree/main/PC-Agent.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。