arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.00554cs.LGcs.AI
收藏

使用提示式 LLM 对激活探针进行红队测试

Red-teaming Activation Probes using Prompted LLMs

Independent(独立研究者)

Phil Blandfort, Robert Graham

AI总结:

提出一种结合迭代反馈与上下文学习的轻量级黑盒红队测试流程,利用现成 LLM 无需微调即可有效发现激活探针在对抗压力下的脆弱性模式,为部署前加固探针提供可操作性见解。

摘要:

激活探针因低成本和低延迟而成为 AI 系统极具吸引力的监视器,但其在现实世界中的鲁棒性仍有待深入研究。我们提出疑问:在现实的无标签黑盒对抗压力下会出现哪些失效模式,以及如何以最小的代价发现它们?我们提出了一种轻量级的黑盒红队测试流程,该流程通过迭代反馈和上下文学习(ICL)封装现成的 LLM,且无需微调、梯度或架构访问权限。通过对高风险交互探针进行案例研究,我们表明该方法有助于发现关于 SOTA 探针的有价值见解。我们的分析揭示了可解释的脆弱性模式(例如,法律术语导致的 FP;平淡的程序性语气导致的 FN),以及在场景约束攻击下有所减少但仍持续存在的漏洞。这些结果表明,简单的提示式红队测试脚手架能够在部署前预测失效模式,并可能产生有前景且具可操作性的见解以加固未来的探针。

英文摘要:

Activation probes are attractive monitors for AI systems due to low cost and latency, but their real-world robustness remains underexplored. We ask: What failure modes arise under realistic, black-box adversarial pressure, and how can we surface them with minimal effort? We present a lightweight black-box red-teaming procedure that wraps an off-the-shelf LLM with iterative feedback and in-context learning (ICL), and requires no fine-tuning, gradients, or architectural access. Running a case study with probes for high-stakes interactions, we show that our approach can help discover valuable insights about a SOTA probe. Our analysis uncovers interpretable brittleness patterns (e.g., legalese-induced FPs; bland procedural tone FNs) and reduced but persistent vulnerabilities under scenario-constraint attacks. These results suggest that simple prompted red-teaming scaffolding can anticipate failure patterns before deployment and might yield promising, actionable insights to harden future probes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑