运行时技能审计:针对智能体技能安全的目标运行时探测
Runtime Skill Audit: Targeted Runtime Probing for Agent Skill Security
AI总结:
提出运行时技能审计(RSA)动态分析方法,通过目标运行时条件探测技能行为,在100个技能上达到90.0%准确率,优于静态基线。
摘要:
智能体技能让LLM智能体能够复用指令、资源、工具和工作流,但也为恶意行为提供了新的隐藏场所。一个技能在其文档或代码中可能看起来无害,但只有在与特定用户请求、本地资产、持久状态或多步骤工具交互调用时才会变得有害。这使得纯静态审查变得脆弱。我们提出运行时技能审计(RSA),一种动态分析方法,通过询问技能介导的智能体在目标运行时条件下实际做了什么来审计技能。RSA不是用相同的通用任务测试每个技能,而是分析风险相关接口,准备执行上下文以触发这些接口,并根据产生的跟踪证据分配安全标签。我们在OpenClaw上实现RSA,并在100个技能上针对代表性静态基线进行评估。RSA达到90.0%的准确率,88.0%的真阳性率和8.0%的假阳性率,比最佳静态基线提高13.0个百分点。在自进化攻击下,静态检测器在一两轮后崩溃,而RSA在每轮中持续检测出19-20个恶意技能。
英文摘要:
Agent skills let LLM agents reuse instructions, resources, tools, and workflows, but they also create a new place for malicious behavior to hide. A skill may look benign in its documentation or code while becoming harmful only when it is invoked with particular user requests, local assets, persistent state, or multi-step tool interactions. This makes purely static vetting brittle. We present Runtime Skill Audit (RSA), a dynamic analysis method that audits skills by asking what the skill-mediated agent actually does under targeted runtime conditions. Instead of testing every skill with the same generic tasks, RSA profiles risk-relevant interfaces, prepares the execution context needed to exercise them, and assigns security labels from the resulting trace evidence. We instantiate RSA on OpenClaw and evaluate it on 100 skills against representative static baselines. RSA achieves 90.0\% accuracy with an 88.0\% true positive rate and an 8.0\% false positive rate, improving accuracy by 13.0 percentage points over the best static baseline. Under self-evolving attacks, static detectors collapse after one or two rounds, while RSA continues to detect 19--20 out of 20 malicious skills across rounds.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。