arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.13889cs.AIcs.IR

RATED:基于执行动态的可靠性感知AI智能体工具检索

RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics

Texas Tech University(德克萨斯理工大学)

Weiliang Chen, Song Liao

AI总结:

本研究针对AI智能体工具检索忽略执行中工具可用性的问题,提出RATED框架,结合双编码器语义匹配与上下文博弈模块,在EcoAgent基准上实现了检索性能与可靠性的显著提升。

摘要:

随着AI智能体越来越依赖外部工具,准确的工具检索对实现可靠行为至关重要。现有方法优化语义相关性,但在很大程度上忽略了执行过程中的工具可用性。为解决这一差距,我们提出RATED(Reliability-Aware Tool Retrieval via Execution Dynamics,基于执行动态的可靠性感知工具检索),这是一种结合鲁棒语义匹配与动态可靠性适配的检索框架。RATED采用经过对比微调的冻结双编码器进行语义匹配,以及轻量型上下文博弈模块,该模块基于实时执行反馈对候选工具进行重新排序。此设计可在无需昂贵模型重新训练的情况下快速适配运行时工具故障。我们引入EcoAgent基准,包含5172个真实世界工具和超过210000个查询-工具对,用于离线检索和动态可靠性评估。实验结果表明,在真实世界查询上,RATED相比ToolLLM API检索器实现了52.2%的相对MRR提升;在故障发生率为30%的严重API中断场景下,将故障选择率从86.2%降至2.9%,且在仅编码推理基础上产生的延迟开销可忽略,不足1毫秒。代码和数据集可在指定URL获取。

英文摘要:

As AI agents increasingly rely on external tools, accurate retrieval is essential for reliable behavior. Existing methods optimize semantic relevance but largely ignore tool availability during execution. To address this gap, we propose RATED (Reliability-Aware Tool Retrieval via Execution Dynamics), a retrieval framework that combines robust semantic matching with dynamic reliability adaptation. RATED employs a frozen dual-encoder with contrastive fine-tuning for semantic matching, and a lightweight contextual bandit module that re-ranks candidates based on real-time execution feedback. This design enables rapid adaptation to runtime tool failures without expensive model retraining. We introduce EcoAgent, a benchmark of 5,172 real-world tools and over 210,000 query-tool pairs for offline retrieval and dynamic reliability evaluation. Experimental results demonstrate that RATED achieves a 52.2% relative MRR improvement over the ToolLLM API retriever on real-world queries, and reduces fault selection rate from 86.2% to 2.9% under severe API outage scenarios with a 30% failure rate, while incurring negligible latency overhead of less than 1 ms over encode-only inference. The code and dataset are available at https://github.com/Elliotes/EcoAgent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑