arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

当工具支持的技能检索失效时:可执行能力检索中的源风格崩溃

When Tool-Backed Skill Retrieval Fails: Source-Style Collapse in Executable Capability Retrieval

Yiqi Liu, Joseph James, Yang Wang, Chenghao Xiao, Chenghua Lin

arXiv 2608.16502首次发表:更新:

发表机构

University of Manchester; University of Sheffield; Shanghai University of Finance and Economics(曼彻斯特大学; 谢菲尔德大学; 上海财经大学)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

该研究针对可执行能力检索中出现的源风格崩溃问题,提出源感知路由方法ToolScout,利用TF-IDF指纹提升检索覆盖率,验证了该方法的有效性。

AI 中文摘要

大规模智能体越来越依赖检索来获取外部能力,我们研究结构化工具和API中的这一检索门,这是一类可测量的工具支持的可执行技能,必须先将其呈现给智能体,智能体才能进行规划、整合或行动。在这种设置下,即使能力语料库固定,检索层也可能无声失效:在ToolRet上,一个在某一特定源切片上微调的检索器,在同一基准的另一特定源切片上出现崩溃,尽管FT-1100与黄金工具的词汇重叠度更高,它仍会出现该情况。我们将这种失效模式称为源风格崩溃。查询侧TF-IDF指纹能比语义或基于长度的代理更好地标记微调检索器可能失效的源风格,为固定工具语料库上的不匹配提供了一种廉价信号。我们提出了ToolScout,一种源感知路由方法,将该信号用作路由守卫:在包含4996个查询的混合流上,基于TF-IDF的路由将覆盖率从22.3%提升至86.1%;在5个崩溃源上,20个匹配示例将覆盖率加权的全局Top-1代理从1.3%提升至53.9%。当工具被重新渲染为可执行技能卡片时,相同的失效和路由行为依然存在,这排除了原始API模式格式是唯一原因的可能。

英文摘要

Large-scale agents increasingly rely on retrieval to access external capabilities. We study this retrieval gate in structured tools and APIs, a measurable class of tool-backed executable skills that must be surfaced before an agent can plan, incorporate, or act. In this setting the retrieval layer can silently fail even when the capability corpus is fixed: on ToolRet, a retriever fine-tuned on one source-specific slice collapses on another source-specific slice of the same benchmark, with FT-1100 despite its higher lexical overlap with the gold tools. We call this failure mode source-style collapse. Query-side TF-IDF fingerprints flag source styles on which the fine-tuned retriever is likely to fail better than semantic or length-based proxies, giving a cheap signal for mismatch over a fixed tool corpus. We propose ToolScout, a source-aware routing method that uses this signal as a routing guard: on the mixed 4,996-query stream, TF-IDF-based routing raises coverage from 22.3% to 86.1%, and across five collapsed sources 20 matched examples raise the coverage-weighted global top-1 proxy from 1.3% to 53.9%. The same failure and routing behaviors persist when tools are rerendered as executable skill cards, which rules out raw API-schema format as the sole cause.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑