arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.10356cs.CL
收藏

自主持续学习用于计算机使用智能体的环境适应

Autonomous Continual Learning for Environment Adaptation of Computer-Use Agents

The Ohio State University(俄亥俄州立大学);University of California, Berkeley(加州大学伯克利分校)

Tianci Xue, Zeyi Liao, Tianneng Shi, Zilu Wang, Kai Zhang, Dawn Song, Yu Su, Huan Sun

AI总结:

本文提出ACuRL框架,通过自主生成任务实现持续学习,无需人工标注数据,在不同环境中实现3-29%的性能提升,且避免灾难性遗忘。

摘要:

现实中的数字环境高度多样且动态,导致智能体频繁遇到未见过的环境和分布偏移,持续学习对计算机使用智能体(CUAs)至关重要。然而,关键挑战在于在不依赖昂贵人工标注的情况下获取高质量且环境相关的训练数据。本文引入ACuRL,一种自主课程强化学习框架,可使智能体持续适应特定环境,无需人工数据。智能体首先探索环境以获取初始经验。在后续迭代训练中,课程任务生成器利用这些经验以及前一次迭代的反馈,合成新的任务以适应智能体当前能力。为提供可靠的奖励信号,我们引入CUAJudge,一种稳健的自动评估器,与人类判断的同意率达93%。实证结果表明,我们的方法有效实现了环境内和跨环境的持续学习,在目标环境中获得3-29%的绝对性能提升,同时在其他环境中避免灾难性遗忘。我们还显示,它能缓解环境变化(如版本更新、平台迁移和分辨率偏移)下的性能下降。进一步分析显示高度稀疏的更新(例如,仅20%的参数),这有助于解释有效且稳健的适应。

英文摘要:

Real-world digital environments are highly diverse and dynamic. These characteristics cause agents to frequently encounter unseen environments and distribution shifts, making continual learning in such environments essential for computer-use agents (CUAs). However, a key challenge lies in obtaining high-quality and environment-grounded training data without relying on costly human annotation. In this work, we introduce ACuRL, an Autonomous Curriculum Reinforcement Learning framework that continually adapts agents to specific environments with zero human data. The agent first explores an environment to acquire initial experiences. During subsequent iterative training, a curriculum task generator leverages these experiences together with feedback from the previous iteration to synthesize new tasks tailored for the agent's current capabilities. To provide reliable reward signals, we introduce CUAJudge, a robust automatic evaluator for CUAs that achieves 93% agreement with human judgments. Empirically, our method effectively enables both intra-environment and cross-environment continual learning, yielding 3-29% absolute performance gains on the target environments without catastrophic forgetting on others. We also show that it can mitigate performance degradation under environment changes (e.g., version updates, platform migration, and resolution shifts). Further analyses show highly sparse updates (e.g., only 20% parameters), which helps explain the effective and robust adaptation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑