arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2509.02444cs.AIcs.CLcs.CVcs.HC
收藏

AppCopilot:迈向通用、准确、长时程且高效的移动智能体

AppCopilot: Toward General, Accurate, Long-Horizon, and Efficient Mobile Agent

Jingru Fan, Yufan Dang, Jingyao Wu, Huatao Li, Runde Yang, Xiyuan Yang, Yuheng Wang, Chen Qian

AI总结:

本文提出AppCopilot,一个多模态多智能体通用移动智能体,通过全流程管道和分层推理协作,解决泛化、精度、长时程与效率四大核心挑战。

摘要:

随着大语言模型和多模态模型的快速发展,移动智能体领域虽然蓬勃发展,但尚未在基本挑战上达成共识。本文指出了移动智能体要实现实用且可扩展的影响所必须解决的四个核心问题:(1)跨任务、跨应用和跨设备的泛化能力;(2)准确性,特别是屏幕交互和点击目标的精准定位;(3)长时程能力,以支持持续的多步骤目标;(4)效率,特别是在资源受限设备上的高性能运行时。我们提出了AppCopilot,一个跨应用运行的多模态、多智能体、通用型移动智能体。AppCopilot通过一个涵盖数据收集、训练、微调、高效推理以及PC/移动端应用的全流程端到端管道来落实这一理念。在模型层,它集成了具有稳健中英文支持的多模态基础模型。在推理与控制层,它结合了思维链推理、层次化任务规划与分解,以及多智能体协作。在执行层,它实现了经验适应、语音交互、函数调用、跨应用与跨设备编排,以及对移动应用的全方位支持。系统设计融入了基于性能剖析的优化,以在异构硬件上优化延迟和内存占用。实验上,AppCopilot在四个维度上取得了显著改进:更强的泛化能力、更高的屏幕操作精度、更可靠的长时程任务完成度,以及更快、更资源高效的运行时。通过阐明一个连贯的立场和一个从数据收集、训练到微调和高效推理形成闭环的参考架构,本文为通用型移动智能体提供了具体路线图和可操作的指导。

英文摘要:

With the raid evolution of large language models and multimodal models, the mobile-agent landscape has proliferated without converging on the fundamental challenges. This paper identifies four core problems that should be solved for mobile agents to deliver practical, scalable impact: (1) generalization across tasks, APPs, and devices; (2) accuracy, specifically precise on-screen interaction and click targeting; (3) long-horizon capability for sustained, multi-step goals; and (4) efficiency, specifically high-performance runtime on resource-constrained devices. We present AppCopilot, a multimodal, multi-agent, general-purpose mobile agent that operates across applications. AppCopilot operationalizes this position through an end-to-end pipeline spanning data collection, training, finetuning, efficient inference, and PC/mobile application. At the model layer, it integrates multimodal foundation models with robust Chinese-English support. At the reasoning and control layer, it combines chain-of-thought reasoning, hierarchical task planning and decomposition, and multi-agent collaboration. At the execution layer, it enables experiential adaptation, voice interaction, function calling, cross-APP and cross-device orchestration, and comprehensive mobile APP support. The system design incorporates profiling-driven optimization for latency and memory across heterogeneous hardware. Empirically, AppCopilot achieves significant improvements on four dimensions: stronger generalization, higher precision of on screen actions, more reliable long horizon task completion, and faster, more resource efficient runtime. By articulating a cohesive position and a reference architecture that closes the loop from data collection, training to finetuning and efficient inference, this paper offers a concrete roadmap for general purpose mobile agent and provides actionable guidance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑