无大纲学习：任务无关的环境预处理

中文摘要 AI 辅助

在大型语言模型（LLM）代理应对新环境中的任务之前，它可以检查可用的语料库和工具，并构建可重用的资源，如索引、脚本或程序性指导。然而，大多数自动化适应方法依赖于任务示例、轨迹或评估反馈来决定构建什么。现有的任务无关方法避免了这种监督，但预先承诺针对特定类型环境采用某种准备策略。我们研究了一个更开放的场景：代理能否在没有大纲的情况下学习一个陌生的环境，即在测试之前且不了解下游任务分布的情况下，并选择如何准备它？我们形式化了任务无关的环境预处理，其中学习系统在预算内探索环境，并为固定的求解器生成工件。我们比较了无辅助和带有存档的元代理与固定的合成练习和语料库处理方法在六个异构基准上的表现。一种元代理变体在五个基准上达到了最高的Avg@3奖励，而固定的语料库处理在最大的语料库基准上仍然表现最佳。更大的学习预算并不能可靠地提高下游奖励。然而，学习到的工件减少了达到给定分数所需的测试时采样，展示了可重用的准备如何将计算从重复的测试时尝试转移到任务前学习阶段。

英文摘要

Before an LLM agent tackles tasks in a new environment, it can inspect available corpora and tools and construct reusable resources such as indices, scripts, or procedural guidance. Most automated adaptation methods, however, rely on task examples, trajectories, or evaluation feedback to decide what to build. Existing task-agnostic approaches avoid this supervision but commit in advance to a preparation strategy for a particular type of environment. We study a more open-ended setting: can an agent study an unfamiliar environment without a syllabus, i.e. before test time and without knowledge of the downstream task distribution, and choose how to prepare it? We formalize task-agnostic environment preprocessing, in which a studying system explores an environment under a budget and produces artifacts for a frozen solver. We compare unaided and archive-equipped meta-agents with fixed synthetic-practice and corpus-processing methods across six heterogeneous benchmarks. A meta-agent variant achieves the highest Avg@3 reward on five benchmarks, while fixed corpus processing remains best on the largest corpus benchmark. Larger study budgets do not reliably improve downstream reward. Nevertheless, studied artifacts reduce the test-time sampling needed to reach a given score, demonstrating how reusable preparation can shift computation from repeated test-time attempts to a pre-task study phase.

发表机构

• Scale AI
• University of Maryland, College Park（马里兰大学学院公园分校）

机构由 AI 辅助整理，请以论文原文为准。