Real-world digital environments are highly diverse and dynamic. These characteristics cause agents to frequently encounter unseen environments and distribution shifts, making continual learning in such environments essential for computer-use agents (CUAs). However, a key challenge lies in obtaining high-quality and environment-grounded training data without relying on costly human annotation. In this work, we introduce ACuRL, an Autonomous Curriculum Reinforcement Learning framework that continually adapts agents to specific environments with zero human data. The agent first explores an environment to acquire initial experiences. During subsequent iterative training, a curriculum task generator leverages these experiences together with feedback from the previous iteration to synthesize new tasks tailored for the agent's current capabilities. To provide reliable reward signals, we introduce CUAJudge, a robust automatic evaluator for CUAs that achieves 93% agreement with human judgments. Empirically, our method effectively enables both intra-environment and cross-environment continual learning, yielding 3-29% absolute performance gains on the target environments without catastrophic forgetting on others. We also show that it can mitigate performance degradation under environment changes (e.g., version updates, platform migration, and resolution shifts). Further analyses show highly sparse updates (e.g., only 20% parameters), which helps explain the effective and robust adaptation.
Continual Machine Reading Comprehension via Uncertainty-aware Fixed Memory and Adversarial Domain Adaptation
Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Hebei University of Science and Technology(河北科技大学)
TRACE: Discovering Task-Specific Parameter via Adaptation-Aware Probing for Continual Fine-Tuning
College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);College of Software, Jilin University(吉林大学软件学院);Fudan University(复旦大学);School of Mathematical and Computer Sciences, Heriot-Watt University(赫瑞-瓦特大学数学与计算机科学学院);Department of Information Engineering and Computer Science, University of Trento(特伦托大学信息工程与计算机科学系)
CTTA-T: Continual Test-Time Adaptation for Text Understanding via Teacher-Student with a Domain-aware and Generalized Teacher
College of Computer Science and Technology, National University of Defense Technology(计算机科学与技术学院,国防科技大学);The Hong Kong University of Science and Technology(香港科技大学)
Efficient Continual Learning in Neural Machine Translation: A Low-Rank Adaptation Approach
Pattern Recognition and Human Language Technology, Universitat Politècnica de València(模式识别与人类语言技术,瓦伦西亚理工大学);Valencian Graduate School and Research Network of Artificial Intelligence, Universitat Politècnica de València(瓦伦西亚人工智能研究生学院与研究网络,瓦伦西亚理工大学)
Unifying Detection and Adaptation in Task-Free Continual Learning
School of Control Science and Engineering, Shandong University(山东大学控制科学与工程学院);The Hong Kong University of Science and Technology(香港科技大学);Shandong University(山东大学);Qilu Hospital of Shandong University(山东大学齐鲁医院)
HyLoVQA: Dynamic Hypernetwork-Generated Low-Rank Adaptation for Continual Visual Question Answering
School of Computer Science, Hubei University, Wuhan 430062, China(湖北大学计算机学院,武汉430062,中国);Hubei Key Laboratory of Big Data Intelligent Analysis and Application (Hubei University), Wuhan 430062, China(湖北省大数据智能分析与应用重点实验室(湖北大学),武汉430062,中国);Key Laboratory of Intelligent Sensing System and Security (Hubei University), Ministry of Education, Wuhan 430062, China(智能感知系统与安全重点实验室(湖北大学),教育部,武汉430062,中国)
CASCADE: Case-Based Continual Adaptation for Large Language Models During Deployment
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Jilin University(吉林大学知识驱动人机智能工程研究中心);International Center of Future Science, Jilin University(吉林大学未来科学国际中心);Department of Informatics, King’s College London(伦敦国王学院信息学院);The Alan Turing Institute(艾伦·图灵研究所);AI Centre, Department of Computer Science, UCL(UCL计算机科学系人工智能中心)