arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.15964cs.AIcs.SE
收藏

虽小但强大:测试套件作为自进化智能体的改进信号

Small but Mighty: Test Suites as Improvement Signals for Self-Evolving Agents

Virginia Tech(弗吉尼亚理工大学);NEC Laboratories America(美国NEC实验室)

Yutong Cheng, Wei Cheng, Wenchao Yu, Dawei Zhou, Haifeng Chen

AI总结:

本研究提出TeSearch方法,通过质量-多样性进化算法搜索小型且强有效的测试套件,作为自进化智能体的改进信号,可高效拒绝多数改变行为的重写,性能优于现有方法。

摘要:

自进化智能体会重写自身的测试框架,且必须决定保留哪些重写内容,因此其改进信号是重写必须通过的检查项。通过智能体滚出验证重写内容成本高昂且具有主观性,而仓库自身测试这类可执行检查项过于稀疏,无法捕捉改变行为的重写内容,仅能拒绝其中四分之一。我们转而将测试反馈作为可执行信号,采用质量-多样性进化算法搜索既小又强的测试套件,使信号密集且具有信息性。TeSearch 将套件大小视为行为描述符,因此在每个大小层级都追求质量,可得到小型且强大的套件,并在搜索过程中学习转换核,以对该描述符上作为执行器的大语言模型(LLM)进行去噪。在129个真实模块和三种主干模型上,与包括LLM驱动进化搜索在内的所有方法相比,TeSearch生成的套件质量最高,且是所有基于LLM的方法中最小的。将其用作自进化智能体的改进信号时,它能拒绝59.9%的改变行为的重写内容,而仓库测试的这一比例仅为25.3%,误报率为4.0%,每次决策耗时3.5秒。代码和数据可在该https URL获取。

英文摘要:

A self-evolving agent rewrites its own harness and must decide which rewrites to keep, so its improvement signal is the check a rewrite has to pass. Verifying a rewrite by agent rollout is expensive and subjective, while executable checks such as the repository's own tests are too sparse to catch behavior-changing rewrites, rejecting only a quarter of them. We instead take test feedback as the executable signal, and search for test suites that are both small and strong with a quality-diversity evolutionary algorithm, so that the signal is dense and informative. TeSearch treats suite size as a behavior descriptor, so that quality is pursued at every size and a small, strong suite is reachable, and learns a transition kernel during the search that denoises the LLM as an actuator on that descriptor. Across 129 real-world modules and three backbones, TeSearch delivers the highest-quality suites of any method compared, LLM-driven evolutionary search included, and the smallest of any LLM-based one. Used as the improvement signal of a self-evolving agent, it rejects 59.9% of behavior-changing rewrites against 25.3% for the repository's tests, at a 4.0% false-alarm rate and 3.5 s per decision. The code and data are available at https://github.com/ccsnow127/TeSearch.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑