UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agents
Multimedia Laboratory (MMLab), The Chinese University of Hong Kong(香港中文大学多媒体实验室);Shenzhen Loop Area Institute(深圳河套学院);Shanghai AI Lab(上海人工智能实验室);vivo AI Lab(vivo人工智能实验室);Princeton University(普林斯顿大学)
Han Xiao, Guozhi Wang, Hao Wang, Shilong Liu, Yuxiang Chai, Yue Pan, Yufeng Zhou, Xiaoxin Chen, Yafei Wen, Hongsheng Li
Online Reinforcement Learning (RL) offers a promising paradigm for enhancing GUI agents through direct environment interaction. However, its effectiveness is severely hindered by inefficient credit assignment in long-horizon tasks and repetitive errors across tasks due to the lack of experience transfer. To address these challenges, we propose UI-Mem, a novel framework that enhances GUI online RL with a Hierarchical Experience Memory. Unlike traditional replay buffers, our memory accumulates structured knowledge, including high-level workflows, subtask skills, and failure patterns. These experiences are stored as parameterized templates that enable cross-task and cross-application transfer. To effectively integrate memory guidance into online RL, we introduce Stratified Group Sampling, which injects varying levels of guidance across trajectories within each rollout group to maintain outcome diversity, driving the unguided policy toward internalizing guided behaviors. Furthermore, a Self-Evolving Loop continuously abstracts novel strategies and errors to keep the memory aligned with the agent's evolving policy. Experiments on online GUI benchmarks demonstrate that UI-Mem significantly outperforms traditional RL baselines and static reuse strategies, with strong generalization to unseen applications. Project page: https://ui-mem.github.io
AtlasVA: Self-Evolving Visual Skill Memory for Teacher-Free VLM Agents
Ant Group(蚂蚁集团);University of Science and Technology of China(中国科学技术大学);Westlake University(西湖大学);University of Michigan - Ann Arbor(密歇根大学-安娜堡分校);Sun Yat-sen University(中山大学)
Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Peng Cheng Laboratory(鹏城实验室);Wuhan AI Research(武汉人工智能研究院)
VideoEvolve: Co-Evolving Memory and Retrieval for Long Video Understanding
Alibaba Group(阿里巴巴集团);University of Science and Technology of China(中国科学技术大学);Shanghai Jiao Tong University(上海交通大学);University of Chinese Academy of Sciences(中国科学院大学);Beihang University(北京航空航天大学)
Echo-Infinity: Learning Evolving Memory for Real-Time Infinite Video Generation
The Chinese University of Hong Kong(香港中文大学);Joy Future Academy, JD(京东探索研究院);The Hong Kong University of Science and Technology(香港科学与技术大学);Tsinghua University(清华大学);The University of Hong Kong(香港大学);University of Science and Technology of China(中国科学技术大学)
Robo-Cortex: A Self-Evolving Embodied Agent via Dual-Grain Cognitive Memory and Autonomous Knowledge Induction
The Hong Kong University of Science and Technology(香港科学与技术大学);X-Humanoid;Institute of Automation, CAS(中国科学院自动化研究所);Beijing University of Aeronautics and Astronautics(北京航空航天大学)