arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.01241cs.CR
收藏

窥视屏障之后:大型语言模型中的屏障识别

Peering Behind the Shield: Guardrail Identification in Large Language Models

CISPA Helmholtz Center for Information Security(CISPA 赫尔姆霍兹信息安全中心);Institute of Science Tokyo(东京科学大学)

Ziqing Yang, Yixin Wu, Rui Wen, Michael Backes, Yang Zhang

AI总结:

本文提出AP-Test方法,通过对抗性提示检测黑盒AI代理中的屏障,解决安全对齐LLM和缺乏决策策略的问题,实验显示其在多种场景下实现完美分类准确率。

摘要:

随着大型语言模型(LLMs)的快速普及,对话式AI代理已广泛应用于现实世界。为提高安全性,这些代理通常配备屏障以调节有害内容。识别代理中的屏障对对手了解系统并设计针对屏障的攻击至关重要。本文介绍了AP-Test,一种新的方法,利用针对屏障的对抗性提示来检测黑盒AI代理中部署的屏障身份。我们的方法解决了该任务中的关键挑战,包括安全对齐LLM和其他屏障的影响,以及缺乏系统决策策略的问题。AP-Test采用两种互补的测试策略,输入和输出屏障测试,以及一个新的指标,匹配分数,以实现稳健的识别。在多样化的代理和四个开源屏障上的实验表明,AP-Test在多种场景中实现了完美的分类准确率。消融研究进一步强调了我们提出组件的必要性。我们的发现揭示了现实世界AI系统中屏障识别的实用路径。

英文摘要:

With the rapid adoption of large language models (LLMs), conversational AI agents have become widely deployed across real-world applications. To enhance safety, these agents are often equipped with guardrails that moderate harmful content. Identifying the guardrails in an agent thus becomes critical for adversaries to understand the system and design guard-specific attacks. In this work, we introduce AP-Test, a novel approach that leverages guard-specific adversarial prompts to detect the identity of guardrails deployed in black-box AI agents. Our method addresses key challenges in this task, including the influence of safety-aligned LLMs and other guardrails, as well as a lack of principled decision-making strategies. AP-Test employs two complementary testing strategies, input and output guard tests, and a new metric, match score, to enable robust identification. Experiments across diverse agents and four open-source guardrails demonstrate that AP-Test achieves perfect classification accuracy in multiple scenarios. Ablation studies further highlight the necessity of our proposed components. Our findings reveal a practical path toward guardrail identification in real-world AI systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑