SkillSafetyBench：在技能面攻击表面下评估智能体安全性
SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfaces
AI总结：
提出SkillSafetyBench基准，通过155个对抗案例评估大语言模型智能体在技能、本地工件和执行环境文件等非用户攻击下的安全失败模式。
摘要：
可复用技能正成为扩展大语言模型智能体的常见接口，它将程序性指导与对文件、工具、内存和执行环境的访问打包在一起。然而，这种模块化引入了现有安全评估大多忽略的攻击面：即使用户请求是良性的，不安全的影响可能存在于技能指导、本地工件或执行环境文件中，这些会引导智能体采取不安全行为。我们提出了SkillSafetyBench，一个可运行的基准，用于评估此类技能中介的安全失败。SkillSafetyBench包含跨47个任务、6个风险领域和30个安全类别的155个对抗案例，每个案例都使用特定于案例的基于规则的验证器进行评估。使用多个CLI智能体和模型后端的实验表明，非用户攻击可以一致地诱导不安全行为，在不同领域、攻击方法和脚手架-模型配对中表现出不同的失败模式。我们的发现表明，智能体安全性不仅取决于模型级别的对齐，还取决于智能体如何解释技能、信任工作流上下文以及通过可执行环境采取行动。
英文摘要：
Reusable skills are becoming a common interface for extending large language model agents, packaging procedural guidance with access to files, tools, memory, and execution environments. However, this modularity introduces attack surfaces that are largely missed by existing safety evaluations: even when the user request is benign, unsafe influence may reside in skill guidance, local artifacts, or execution-environment files that steer the agent toward unsafe actions. We present SkillSafetyBench, a runnable benchmark for evaluating such skill-facing safety failures. SkillSafetyBench includes 155 adversarial cases across 47 tasks, 6 risk domains, and 30 safety categories, each evaluated with a case-specific rule-based verifier. Experiments with multiple CLI agents and model backends show that non-user attacks can consistently induce unsafe behavior, with distinct failure patterns across domains, attack methods, and scaffold-model pairings. Our findings suggest that agent safety depends not only on model-level alignment, but also on how agents interpret skills, trust workflow context, and act through executable environments. The complete benchmark is available at https://github.com/AI45Lab/skill-safety-bench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
