HarnessCompass：引导自动 harness 进化以生成可泛化且有效的智能体 harness
HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses
浏览论文内容
中文总结 AI 辅助
HarnessCompass 是一种自动 harness 进化框架，通过约束进化、主动反馈和组件式优化，在 SWE-bench Verified 上用 GPT-5.4 使 Pass@1 提升至 66%，且泛化能力优于现有方法。
中文摘要 AI 辅助
harness 设计通过塑造大语言模型（LLM）在可执行环境中的感知、推理和行动方式，对智能体性能起着关键作用。近期研究提出了自动 harness 进化，该方法通过智能体与环境的交互迭代改进 harness。然而，现有方法常过拟合于进化任务，仅依赖轨迹衍生的信号，且联合优化 harness 组件，导致组件间相互干扰。我们提出 HarnessCompass，这是一种新型自动 harness 进化框架，围绕约束进化、主动反馈和组件式优化构建。HarnessCompass 首先对进化实施全局约束，将修改限制为任务无关的 harness 变更，使其能泛化到进化任务之外。随后，它用智能体关于 harness 使用的主动第一人称反馈增强轨迹衍生的证据，为进化提供更丰富的信号。最后，它在整合不同 harness 组件为统一 harness 前，对各组件的优化进行解耦，减少组件间干扰的同时保留组件协同效应。在 SWE-bench Verified 上使用 GPT-5.4 时，HarnessCompass 仅在 5 次进化迭代中就将 Pass@1 从 54% 提升至 66%，在有效性和进化效率上均优于 AHE。此外，进化后的 harness 能有效迁移到未见过的任务和其他模型，展现出比现有自动 harness 进化方法强得多的泛化能力。
英文摘要
Harness design plays a critical role in agent performance by shaping how large language models (LLMs) perceive, reason over, and act within executable environments. Recent work has proposed automatic harness evolution, which iteratively improves the harness from agent--environment interactions. However, existing methods often overfit to the evolution tasks, rely exclusively on trajectory-derived signals, and optimize harness components jointly, causing interference across components. We propose HarnessCompass, a novel automatic harness evolution framework built around constrained evolution, proactive feedback, and component-wise optimization. HarnessCompass first enforces global constraints on evolution, restricting modifications to task-agnostic harness changes that generalize beyond the evolution tasks. It then augments trajectory-derived evidence with proactive first-person feedback from the agent about harness usage, yielding richer signals for evolution. Finally, it decouples the optimization of different harness components before consolidating them into a unified harness, reducing cross-component interference while preserving component synergy. On SWE-bench Verified with GPT-5.4, HarnessCompass improves Pass@1 from 54\% to 66\% in only 5 evolution iterations, outperforming AHE in both effectiveness and evolution efficiency. In addition, the evolved harness transfers effectively to held-out tasks and other models, demonstrating substantially stronger generalization than prior automatic harness evolution methods.
