arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2504.04809cs.CR
收藏

SEEM:利用黑盒文本攻击操纵工具选择

SEEM: Exploiting Black-Box Text Attacks to Manipulate Tool Selection

New Laboratory of Pattern Recognition (NLPR)(模式识别国家重点实验室);State Key Laboratory of Multimodal Artificial Intelligence Systems(多模态人工智能系统全国重点实验室);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Beijing University of Posts and Telecommunications(北京邮电大学)

Liuji Chen, Hao Gao, Jinghao Zhang, Qiang Liu, Shu Wu, Liang Wang

AI总结:

提出SEEM方法,通过词级和字符级的粗到细扰动,在黑盒条件下增加目标工具被选中的概率,揭示工具选择机制的安全漏洞。

摘要:

工具学习已成为一种强大的辅助机制,扩展了大语言模型(LLM)的能力,使其能够处理需要实时相关性或高精度操作的复杂任务。然而,在这种优势之下隐藏着显著的安全风险。先前的研究主要集中在破坏所调用工具的输出上,而很大程度上忽略了工具选择过程本身的脆弱性。为填补这一空白,我们引入了一种黑盒、基于文本的攻击,该攻击显著增加了目标工具被选中的可能性。我们提出了SEEM,一种在词级和字符级上操作的两级粗到细扰动方法。通过全面的实验,我们表明仅扰动工具的文本信息就能显著提高目标工具在候选工具中被优先考虑和排名更高的概率。我们的发现揭示了工具选择机制中的关键弱点,并为开发防御措施以保护这一关键过程奠定了基础。

英文摘要:

Tool learning has emerged as a powerful auxiliary mechanism that extends the capabilities of large language models (LLMs), enabling them to address complex tasks that demand real-time relevance or high-precision operations. However, beneath this strength lie significant security risks. Prior studies have primarily concentrated on corrupting the outputs of invoked tools, while largely overlooking the vulnerability of the tool selection process itself. To bridge this gap, we introduce a black-box, text-based attack that substantially increases the likelihood of a target tool being selected. We propose SEEM, a two-level coarse-to-fine perturbation method that operates at both the word and character levels. Through comprehensive experiments, we show that merely perturbing the textual information of tools can markedly raise the probability of the target tool being prioritized and ranked higher among candidates. Our findings expose critical weaknesses in the tool selection mechanism and lay the groundwork for developing defenses to secure this essential process.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑