arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.36826cs.AI
收藏

当评判者落后时:用于智能体持续自我改进的共同演化准则与控制机制

When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement

Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology(哈尔滨工业大学社会计算与交互机器人研究中心);Department of Computer Science and Technology, Institute for AI, Tsinghua University(清华大学计算机科学与技术系、人工智能研究院)

Hao Chen, Yukun Yan, Ziyu Han, Ye He, Chuanzhe Guo, Shangzhan Li, Qi Shi, Qingfu Zhu, Maosong Sun, Wanxiang Che

AI总结:

本研究针对智能体控制机制演化中监督信号静态的问题,提出RuHarness框架,通过准则与控制机制共同演化,在多任务及骨干模型上实现优于基线的性能,尤其提升了未见任务的泛化能力。

摘要:

自主智能体越来越依赖围绕大语言模型(LLM)管理交互、记忆、工具及执行的控制机制(harness)。尽管近期研究已开始从执行经验中演化这些控制机制,但用于指导演化的监督信号往往仍相对静态,难以跟踪多轮中不断变化的行为弱点,且可能限制对未见任务的泛化能力。为解决该问题,我们提出RuHarness——一种用于智能体持续自我改进的准则-控制机制共同演化框架。RuHarness持续从执行轨迹中发现并重新评估行为准则,优先处理最相关的未解决弱点,并在保持任务目标固定的前提下,用这些弱点指导针对性的控制机制更新。在多种智能体任务及骨干模型上开展的大量实验表明,RuHarness优于强大的控制机制演化基线,尤其在未见的保留任务上表现突出。进一步分析显示,自适应准则优先级排序可跟踪不断变化的弱点,并在多轮演化中支持更持续的改进与稳定的泛化。

英文摘要:

Autonomous agents increasingly depend on harnesses that govern interaction, memory, tools, and execution around large language models (LLMs). While recent work has begun to evolve these harnesses from execution experience, the supervision used to guide evolution often remains relatively static, making it difficult to track changing behavioral weaknesses across rounds and potentially limiting generalization to unseen tasks. To address this issue, we propose RuHarness, a rubric--harness co-evolution framework for continual agent self-improvement. RuHarness continually discovers and reassesses behavioral criteria from execution trajectories, prioritizes the most relevant unresolved weaknesses, and uses them to guide targeted harness updates while keeping the task objective fixed. Extensive experiments across diverse agent tasks and backbone models show that RuHarness outperforms strong harness-evolution baselines, particularly on unseen held-out tasks. Further analyses show that adaptive rubric prioritization tracks shifting weaknesses and supports more sustained improvement and stable generalization across evolution rounds.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑