arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.16943cs.AIcs.SE
收藏

注意GAP:在LLM代理中,文本安全不转移到工具调用安全

Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents

Independent Researcher(独立研究者)

Arnold Cartagena, Ariane Teixeira

AI总结:

该研究发现LLM代理中文本安全不等同于工具调用安全,引入GAP基准测试揭示两者差异,强调需专门评估工具调用安全。

摘要:

大型语言模型作为代理部署时,越来越多地通过工具调用与外部系统交互——这些动作具有现实世界的影响,而仅靠文本输出无法传达这些影响。然而,安全评估却大多只测量文本层面的拒绝行为,留下了一个关键问题未被回答:在抑制有害文本的对齐方式是否也抑制有害动作?我们引入了GAP基准测试,这是一种系统性的评估框架,用于衡量LLM代理中文本层面安全与工具调用层面安全之间的差异。我们测试了六个前沿模型,涵盖六个受监管的领域(制药、金融、教育、就业、法律和基础设施),每个领域七个 jailbreak 情景,三种系统提示条件(中性、安全强化和工具鼓励),以及两种提示变体,产生17,420个分析就绪的数据点。我们的核心发现是文本安全不转移到工具调用安全。在所有六个模型中,我们观察到实例,其中模型的文本输出拒绝有害请求,同时其工具调用却同时执行禁止的动作——这种差异我们正式称为GAP指标。即使在安全强化的系统提示下,六个模型中仍存在219种情况。系统提示语言对工具调用行为有显著影响:对于最稳健的模型,TC安全率跨度21个百分点,而对于最敏感于提示的模型,跨度57个百分点,其中16个18对的消融比较在Bonferroni校正后仍显著。运行时治理合同在所有六个模型中都减少了信息泄露,但对禁止的工具调用尝试本身没有产生可检测的遏制效果。这些结果表明,仅文本的安全评估不足以评估代理行为,而工具调用安全需要专门的测量和缓解。

英文摘要:

Large language models deployed as agents increasingly interact with external systems through tool calls--actions with real-world consequences that text outputs alone do not carry. Safety evaluations, however, overwhelmingly measure text-level refusal behavior, leaving a critical question unanswered: does alignment that suppresses harmful text also suppress harmful actions? We introduce the GAP benchmark, a systematic evaluation framework that measures divergence between text-level safety and tool-call-level safety in LLM agents. We test six frontier models across six regulated domains (pharmaceutical, financial, educational, employment, legal, and infrastructure), seven jailbreak scenarios per domain, three system prompt conditions (neutral, safety-reinforced, and tool-encouraging), and two prompt variants, producing 17,420 analysis-ready datapoints. Our central finding is that text safety does not transfer to tool-call safety. Across all six models, we observe instances where the model's text output refuses a harmful request while its tool calls simultaneously execute the forbidden action--a divergence we formalize as the GAP metric. Even under safety-reinforced system prompts, 219 such cases persist across all six models. System prompt wording exerts substantial influence on tool-call behavior: TC-safe rates span 21 percentage points for the most robust model and 57 for the most prompt-sensitive, with 16 of 18 pairwise ablation comparisons remaining significant after Bonferroni correction. Runtime governance contracts reduce information leakage in all six models but produce no detectable deterrent effect on forbidden tool-call attempts themselves. These results demonstrate that text-only safety evaluations are insufficient for assessing agent behavior and that tool-call safety requires dedicated measurement and mitigation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑