arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.29320cs.AI

MAGA:基于结构化动作蒸馏的GUI智能体多平台自融合

MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillation

  • Ant Group(蚂蚁集团)

机构由 AI 辅助整理,请以论文原文为准。

Hang Yan, Zhangxuan GU, Beitong Zhou, Jiaxuan Chen, Runze Li, Yusong Hu, Shuheng Shen, Changhua Meng

AI总结:

该研究针对GUI智能体跨平台部署受限问题,提出MAGA方法优化训练信号分配,在8B参数规模下性能优于基准模型,与教师模型表现接近。

AI中文摘要:

基于大语言模型的图形用户界面(GUI)智能体正越来越多地部署在移动、网页和桌面环境中,但现有智能体通常是领域特定的,限制了部署范围和用户体验,这促使人们将专用模型整合为单一跨环境策略。权重合并可直接合并领域特定专家,但在专家意见不一致时会破坏可执行动作;而在线策略蒸馏(OPD)虽能避免冲突的教师监督,但在蒸馏过程中仍平等对待所有响应标记,忽略了动作标记是智能体与环境之间的唯一接口。为解决此问题,本文提出MAGA,它根据结构化动作重新分配训练信号,基于生成动作的正确性抑制不必要或无效的蒸馏信号,将学习重点放在错误动作上;此外,一种仅用于训练的提示优化了领域特定教师提供的监督信号,且不改变学生输入。在两种模型规模下,MAGA均达到最高平均成功率:8B参数规模时比最强基准模型高2.0%,且与教师模型达到几乎相同的平均性能。

英文摘要:

Graphical user interface (GUI) agents based on large language models are increasingly deployed across mobile, web, and desktop environments. However, existing agents are typically domain-specific, limiting the deployment and user experience. This motivates the consolidation of specialized models into a single cross-environment policy. Weight merging directly merges domain-specific experts but can corrupt executable actions under expert disagreement, while on-policy distillation (OPD) avoids conflicting teacher supervision yet still treats all response tokens equally during distillation, ignoring that action tokens are the only interface between the environment and the agent. To address this, We introduce MAGA that re-allocates training signal according to the structured action. Based on the correctness of the generated action, it suppresses unnecessary or invalid distillation signals and focuses learning on erroneous actions. Besides, a training-only hint optimizes the supervision signal provided by domain-specific teachers without changing the student input. Across two model scales, MAGA achieves the highest mean success rate, outperforming the strongest baseline by 2.0% at 8B and achieves almost the same average performance with teachers.

补充信息

相关深度报道

↑