Mobile-Agent-v2：通过多智能体协作实现高效导航的移动设备操作助手
Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration
AI总结：
针对移动设备操作任务中的导航挑战，提出多智能体架构Mobile-Agent-v2，通过规划、决策和反思三个智能体及记忆单元，显著提升了任务完成率。
摘要：
移动设备操作任务正日益成为一种流行的多模态AI应用场景。当前的多模态大语言模型（MLLMs）受限于其训练数据，缺乏作为操作助手有效运作的能力。相反，基于MLLM的智能体通过工具调用增强能力，正逐渐被应用于该场景。然而，移动设备操作任务中的两大主要导航挑战——任务进度导航和焦点内容导航，在现有工作的单智能体架构下变得显著复杂。这是因为过长的token序列和交错的文本-图像数据格式限制了性能。为了有效应对这些导航挑战，我们提出了Mobile-Agent-v2，一种用于移动设备操作协助的多智能体架构。该架构包含三个智能体：规划智能体、决策智能体和反思智能体。规划智能体生成任务进度，使历史操作的导航更加高效。为保留焦点内容，我们设计了一个随任务进度更新的记忆单元。此外，为纠正错误操作，反思智能体观察每次操作的结果并相应处理任何错误。实验结果表明，与Mobile-Agent的单智能体架构相比，Mobile-Agent-v2在任务完成率上实现了超过30%的提升。代码已在https://github.com/X-PLUG/MobileAgent开源。
英文摘要：
Mobile device operation tasks are increasingly becoming a popular multi-modal AI application scenario. Current Multi-modal Large Language Models (MLLMs), constrained by their training data, lack the capability to function effectively as operation assistants. Instead, MLLM-based agents, which enhance capabilities through tool invocation, are gradually being applied to this scenario. However, the two major navigation challenges in mobile device operation tasks, task progress navigation and focus content navigation, are significantly complicated under the single-agent architecture of existing work. This is due to the overly long token sequences and the interleaved text-image data format, which limit performance. To address these navigation challenges effectively, we propose Mobile-Agent-v2, a multi-agent architecture for mobile device operation assistance. The architecture comprises three agents: planning agent, decision agent, and reflection agent. The planning agent generates task progress, making the navigation of history operations more efficient. To retain focus content, we design a memory unit that updates with task progress. Additionally, to correct erroneous operations, the reflection agent observes the outcomes of each operation and handles any mistakes accordingly. Experimental results indicate that Mobile-Agent-v2 achieves over a 30% improvement in task completion compared to the single-agent architecture of Mobile-Agent. The code is open-sourced at https://github.com/X-PLUG/MobileAgent.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
