CGL: 通过强化微调推进连续GUI学习
CGL: Advancing Continual GUI Learning via Reinforcement Fine-Tuning
AI总结：
CGL通过强化微调与监督微调的协同优化，解决GUI连续学习中遗忘旧任务的问题，提出AndroidControl-CL基准验证方法有效性。
摘要：
图形用户界面（GUI）代理受益于多模态大语言模型（MLLM）最近的进展，已取得显著发展。然而，由于GUI应用程序的频繁更新，使GUI连续学习在适应新任务的同时不遗忘旧任务仍是一个开放性问题。在本工作中，我们揭示，尽管监督微调（SFT）有助于快速适应，但往往触发知识覆盖，而强化学习（RL）表现出固有的韧性，能够保护先前交互逻辑免受擦除。基于这一洞察，我们提出了一种连续GUI学习（CGL）框架，通过增强SFT与RL之间的协同作用，动态平衡适应效率与技能保留。具体而言，我们引入了一种由策略熵引导的SFT比例调整机制，以动态控制SFT与RL训练阶段的权重分配。为了解决显式梯度干扰问题，我们进一步开发了一种专门的梯度手术策略。通过将探索性的SFT梯度投影到基于GRPO的锚定梯度上，我们的方法显式地剪裁与GRPO冲突的SFT梯度组件。此外，我们建立了AndroidControl-CL基准，将GUI应用程序划分为不同的任务组，以有效模拟和评估连续GUI学习的性能。实验结果展示了我们提出的CGL框架在连续学习场景中的有效性。基准、代码和模型将公开发布。
英文摘要：
Graphical User Interface (GUI) Agents, benefiting from recent advances in multimodal large language models (MLLM), have achieved significant development. However, due to the frequent updates of GUI applications, adapting to new tasks without forgetting old tasks in GUI continual learning remains an open problem. In this work, we reveal that while Supervised Fine-Tuning (SFT) facilitates fast adaptation, it often triggers knowledge overwriting, whereas Reinforcement Learning (RL) demonstrates an inherent resilience that shields prior interaction logic from erasure. Based on this insight, we propose a \textbf{C}ontinual \textbf{G}UI \textbf{L}earning (CGL) framework that dynamically balances adaptation efficiency and skill retention by enhancing the synergy between SFT and RL. Specifically, we introduce an SFT proportion adjustment mechanism guided by policy entropy to dynamically control the weight allocation between the SFT and RL training phases. To resolve explicit gradient interference, we further develop a specialized gradient surgery strategy. By projecting exploratory SFT gradients onto GRPO-based anchor gradients, our method explicitly clips the components of SFT gradients that conflict with GRPO. On top of that, we establish an AndroidControl-CL benchmark, which divides GUI applications into distinct task groups to effectively simulate and evaluate the performance of continual GUI learning. Experimental results demonstrate the effectiveness of our proposed CGL framework across continual learning scenarios. The benchmark, code, and model will be made publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
