为大型语言模型网页智能体学习简单的测试时环境
Learning Simple Test-Time Environments for LLM Web Agents
浏览论文内容
中文总结 AI 辅助
本研究针对LLM网页智能体在复杂真实环境中性能下降问题，提出测试时环境分解（TTED）方法，通过将复杂环境分解为子模块并利用子环境经验提升组合泛化能力，验证了其在合成与真实基准上的有效性。
中文摘要 AI 辅助
大型语言模型（LLM）智能体在人工构建的环境中展现出卓越的能力，但当迁移至复杂的真实世界场景时，其性能往往会大幅下降。现有研究大多将这种性能退化归因于LLM在多个简单、结构良好的环境组合上的组合泛化差距。在本研究中，我们提出LLM网页智能体可在测试时学习简单的环境观测。具体而言，我们为智能体引入试错步骤，将复杂的环境观测分解为子模块，并实现了一种无标签学习方法——测试时环境分解（TTED），用于在推理过程中借助经验调整智能体行为。我们的实证评估在合成基准和真实基准上均验证了该框架的有效性，结果显示：（1）在较简单的子环境中获得的经验增益可有效组合以提升完整环境中的性能；（2）在子环境上进行测试时训练可显著增强智能体在真实世界网页自动化任务中的组合泛化能力。我们还提供了无标签学习算法设计的关键见解。随着LLM智能体接触到更复杂的环境，我们认为在测试时学习环境分解技能对于实现稳健的真实世界部署至关重要。
英文摘要
Large language model (LLM) agents have demonstrated remarkable proficiency in manually constructed environments, yet their performance frequently collapses when transitioned to complex real-world settings. Existing research largely attribute this degradation to the compositional generalization gaps in LLMs on combinations of multiple simple, well-structured environments. In this work, we propose that LLM web agents can learn simple environment observations at test time. Specifically, we introduce trial steps for agents to decompose a complex environment observation into sub-modules, and implement a label-free learning method, Test-Time Environment Decomposition (TTED), to adapt agent behaviors with experience during inference. Our empirical evaluations demonstrate the framework's efficacy across both synthetic and realistic benchmarks, showing (1) experience gains acquired within simpler sub-environments can be effectively composed to improve performance in the full one, and (2) test-time training on sub-environments can significantly enhance the compositional generalization of agents in real-world web automation tasks. We also provide key insights in the design of the label-free learning algorithm. As more complex environments are accessed by LLM agents, we believe learning environment decomposition skills at test time will be critical for robust real-world deployment.
发表机构
- College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）
- Dept. of Comp. Sci. & Tech., Tsinghua University（清华大学计算机科学与技术系）
- School of Electronic and Information Engineering, Beijing Jiaotong University（北京交通大学电子信息工程学院）
- Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）
- Tongyi Lab, Alibaba Group（阿里巴巴集团通义实验室）
机构由 AI 辅助整理，请以论文原文为准。
