AI 中文总结
本文针对智能体工具检索的风险问题，提出轻量级风险感知重排序框架，通过权衡安全与效用改善相关性-安全 tradeoff，为安全关键部署提供保守操作点。
AI 中文摘要
工具检索决定了LLM智能体针对用户查询或任务可调用哪些外部工具，是执行前的关键安全边界。与文档检索不同，工具检索会暴露可执行动作：对一项任务有用的工具，对另一项任务可能是不必要的或有风险的。然而，现有工具检索方法主要优化语义相关性，安全评估往往关注工具执行后的失败，而非检索阶段引入的风险。本文研究风险感知工具检索，目标是检索有用工具的同时减少高风险工具的暴露。我们在冻结的第一阶段检索器之上提出轻量级重排序框架，该框架分别建模查询条件下的相关性和工具级暴露风险，通过显式参数控制安全与效用的权衡，在ToolGraph上平滑分数，并可选择性应用基于规则的安全约束。为支持检索时的安全评估，我们在UltraTool和Seal-Tools数据集上对6108个工具标注了5个有序风险等级，并定义了衡量前k个结果中高风险工具暴露的指标。在UltraTool和Seal-Tools上的实验表明，我们的方法比仅考虑相关性的检索器和重排序基线方法改善了相关性-安全权衡，经规则过滤的变体为安全关键部署提供了保守操作点。这些发现表明，检索阶段过滤可缩小智能体执行前暴露的候选动作空间，补充下游工具使用的安全保障。代码和补充材料可在该https URL获取。
英文摘要
Tool retrieval determines which external tools are exposed to an LLM agent for a user query or task, making retrieval a critical pre-execution safety boundary. Unlike document retrieval, tool retrieval exposes executable actions: a tool that is useful for one task may be unnecessary or risky for another. However, existing tool-retrieval methods primarily optimize semantic relevance, and safety evaluations often focus on failures after tool execution rather than risks introduced during retrieval. We study risk-aware tool retrieval, where the goal is to retrieve useful tools while reducing exposure to higher-risk tools. We propose a lightweight reranking framework on top of a frozen first-stage retriever. The framework models query-conditioned relevance and tool-level exposure risk separately, combines them through an explicit parameter controlling the tradeoff between safety and utility, smooths scores over a ToolGraph, and optionally applies rule-based safety constraints. To support retrieval-time safety evaluation, we annotate 6,108 tools across UltraTool and Seal-Tools with five ordinal risk levels and define metrics that measure risky-tool exposure in the top-$k$ results. Experiments on UltraTool and Seal-Tools show that our approach improves the relevance--safety tradeoff over relevance-only retrievers and reranking baselines, with the rule-filtered variant providing a conservative operating point for safety-critical deployments. These findings indicate that retrieval-stage filtering can reduce the candidate action space exposed to agents before execution, complementing downstream tool-use safeguards. The code and supplementary materials are available at: https://github.com/qli447/risk-aware-tool-retrieval-release.
CommentsAccepted by CIKM 2026
