AI 中文总结
SkillSentry是基于DSL的技能运行时保障框架，通过初始化、监控引导与迭代优化提升LLM智能体技能执行可靠性，在15项技能上平均提升任务成功率24.1%且降低变异性。
AI 中文摘要
大语言模型（LLM）智能体日益配备各类技能，可通过多步推理和工具使用完成复杂任务。尽管技能提供了可复用的过程性知识，但智能体执行技能时仍可能存在不可靠问题：即使智能体在技能指导下展现出完成任务的能力，由于偏离技能流程或执行步骤出错，也可能在相似任务或多次运行中无法稳定完成，这种不稳定性限制了LLM智能体的实际可靠性。为解决该问题，本文提出SkillSentry，这是一种面向技能的运行时保障框架，基于新的领域特定语言（DSL）构建，用于表示技能执行的运行时指导。SkillSentry通过结合从对应技能文档提取的技能规范，以及从历史成功与失败轨迹挖掘的执行经验，初始化运行时指导；随后嵌入智能体执行循环，在当前指导下监控并引导技能执行，同时利用新收集的轨迹迭代优化指导。本文在两个LLM智能体的15项技能上评估SkillSentry，每个智能体搭配两个主干模型：Claude Code搭配Claude-Haiku-4.5和Claude-Opus-4.6，Codex搭配GPT-5.2和GPT-5.4。结果显示，SkillSentry平均将LLM智能体的任务成功率提升了24.1%，同时降低了多次运行的变异性。
英文摘要
LLM agents are increasingly equipped with skills to perform complex tasks through multi-step reasoning and tool use. Although skills provide reusable procedural knowledge, agents may still execute them unreliably. Even when an agent has demonstrated the capability to complete tasks under the guidance of a skill, it may fail to do so consistently across similar tasks or repeated runs due to deviations from the skill procedure or incorrect execution of individual steps. Such instability limits the practical reliability of LLM agents. To address this problem, we propose SkillSentry, a skill-oriented runtime assurance framework built upon a new domain-specific language (DSL) for representing runtime guidance for skill execution. SkillSentry initializes the runtime guidance by combining a skill specification extracted from the corresponding skill document with execution experience mined from historical successful and failed traces. It then wraps around the agent execution loop to monitor and guide skill execution under the current guidance, while iteratively refining the guidance using newly collected traces. We evaluate SkillSentry on 15 skills across two LLM agents, each paired with two backbone models, i.e., Claude Code with Claude-Haiku-4.5 and Claude-Opus-4.6, and Codex with GPT-5.2 and GPT-5.4. Our results show that SkillSentry improves the task success rate of LLM agents by 24.1% across skills, on average, while exhibiting lower variability across repeated runs.
