打破环境壁垒：演化LLM智能体环境以实现递归自我改进

中文摘要 AI 辅助

许多现实世界任务（例如办公工作流、科学实验）要求LLM智能体与环境反复交互以进行依赖上下文的操作。然而，此类环境通常并非为智能体就绪。首先，信息往往分散且碎片化地分布在整个环境中。其次，环境中的相关证据常常与误导性信息和相互矛盾的版本混杂在一起。第三，环境随时间演化，引入新的噪声和更具挑战性的任务。这些挑战可能大幅降低最先进AI智能体的性能（例如，从83.9%降至57.6%）。为应对这些挑战，我们提出Env-Rethink（一个具有27B后训练模型的系统），该系统支持三项主要能力：（1）自适应构建集合映射（用于组织相关文件）和事件日志（用于情境化跨数据关系）以补充必要上下文；（2）进一步利用后训练模型（通过离线轨迹学习）识别环境中潜在的噪声问题；（3）最终通过虚拟事件历史演化环境，改变环境状态和证据关系，生成更棘手的场景以促进智能体进一步改进。实验表明，Env-Rethink能有效提升下游任务性能（在30个任务上，九个模型的rubric通过率平均提升超过15.1%）。

英文摘要

Many real-world tasks (e.g., office workflows, scientific experimentation) require LLM agents to interact repeatedly with their environments for context-dependent operations. However, such environments are often not agent-ready. First, information is often scattered and fragmented across the environment. Second, relevant evidence in the environment is often mixed with misleading information and conflicting versions. Third, environments evolve over time, introducing new noise and more challenging tasks. These challenges can substantially degrade performance for state-of-the-art AI agents (e.g., from 83.9% to 57.6%). To address these challenges, we propose Env-Rethink (a system with 27B post-trained model) that supports three main capabilities: (1) It adaptively builds Collection Maps (for organizing related files) and Event Logs (for contextualizing cross-data relationships) to supplement necessary context; (2) It further leverages the post-trained model (through offline trajectory learning) to identify underlying noise issues in the environment; (3) It ultimately evolves environments through virtual event histories that alter environmental states and evidence relationships, producing more tricky ones for further agent improvement. Experiments show that Env-Rethink can effectively improve downstream task performance (with a 15.1 percentage-point increase in mean rubric pass rate across nine models on 30 tasks).

发表机构

• Shanghai Jiao Tong University（上海交通大学）
• Tencent Hunyuan（腾讯混元）

机构由 AI 辅助整理，请以论文原文为准。