arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.05832cs.CV
收藏

UI-Mem: 为移动GUI代理在线强化学习中的自演化经验记忆

UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agents

Multimedia Laboratory (MMLab), The Chinese University of Hong Kong(香港中文大学多媒体实验室);Shenzhen Loop Area Institute(深圳河套学院);Shanghai AI Lab(上海人工智能实验室);vivo AI Lab(vivo人工智能实验室);Princeton University(普林斯顿大学)

Han Xiao, Guozhi Wang, Hao Wang, Shilong Liu, Yuxiang Chai, Yue Pan, Yufeng Zhou, Xiaoxin Chen, Yafei Wen, Hongsheng Li

AI总结:

UI-Mem通过自演化经验记忆提升移动GUI代理在线强化学习性能,实现跨任务经验转移与策略优化。

摘要:

在线强化学习(RL)提供了一种通过直接与环境交互来增强GUI代理的有前途的范式。然而,其有效性受到长周期任务中信用分配效率低下和跨任务重复错误的严重阻碍,这主要是由于缺乏经验转移。为了解决这些挑战,我们提出了UI-Mem,一种新的框架,通过分层经验记忆增强GUI在线RL。与传统回放缓冲区不同,我们的记忆积累结构化知识,包括高层次的工作流、子任务技能和失败模式。这些经验被存储为参数化的模板,以实现跨任务和跨应用的转移。为了有效将记忆指导整合到在线RL中,我们引入了分层分组采样,它在每个rollout组内向轨迹注入不同层次的指导,以保持结果多样性,推动无指导策略向内化指导行为发展。此外,一个自演化循环持续抽象新策略和错误,以保持记忆与代理演变策略的一致性。在在线GUI基准测试中的实验表明,UI-Mem显著优于传统RL基线和静态重用策略,对未见应用具有强大的泛化能力。项目页面:https://ui-mem.github.io

英文摘要:

Online Reinforcement Learning (RL) offers a promising paradigm for enhancing GUI agents through direct environment interaction. However, its effectiveness is severely hindered by inefficient credit assignment in long-horizon tasks and repetitive errors across tasks due to the lack of experience transfer. To address these challenges, we propose UI-Mem, a novel framework that enhances GUI online RL with a Hierarchical Experience Memory. Unlike traditional replay buffers, our memory accumulates structured knowledge, including high-level workflows, subtask skills, and failure patterns. These experiences are stored as parameterized templates that enable cross-task and cross-application transfer. To effectively integrate memory guidance into online RL, we introduce Stratified Group Sampling, which injects varying levels of guidance across trajectories within each rollout group to maintain outcome diversity, driving the unguided policy toward internalizing guided behaviors. Furthermore, a Self-Evolving Loop continuously abstracts novel strategies and errors to keep the memory aligned with the agent's evolving policy. Experiments on online GUI benchmarks demonstrate that UI-Mem significantly outperforms traditional RL baselines and static reuse strategies, with strong generalization to unseen applications. Project page: https://ui-mem.github.io

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑