HESP：在本地 LLM 警报分流智能体中分离“探测什么”与“何时停止”
HESP: Separating What to Probe from When to Stop in Local LLM Alert-Triage Agents
浏览论文内容
中文总结 AI 辅助
HESP 是一种将调查过程置于模型之外的控制器，通过选择只读探测和控制器端停止机制，显著提升了小型本地 LLM 在警报分流中的验证完成率，适用于遥测数据不能离开本地的环境。
中文摘要 AI 辅助
安全运营中心接收的警报数量远超分析师所能调查的范围，而无法将遥测数据发送至托管模型的组织必须使用小型开源权重 LLM 在自己的硬件上自动化分流。当前的 LLM 智能体将调查过程交由模型自行决定，而小型本地模型在此任务上表现不佳：它们不断探测却不收敛，从不做出最终判定，或直接忽略真实攻击。在本文中，我们提出了 HESP，一种将调查过程置于模型之外的控制器。HESP 维护一个竞争性解释的台账，根据每次探测的预期信息增益与成本之比选择只读探测，仅接受有当前证据支持的判定，能够自行终止调查，并在每次预测前记录其日志。我们在四项预注册研究中评估了 HESP，使用了来自两个系列的五个开源权重模型（7B 至 72B），在受控分流环境中总计进行了 7,272 次审计会话。利用从无 LLM 运行中统计出的似然表，HESP 将 Qwen2.5-7B 的验证完成率从 0.125 提升至 1.000，与 oracle 表持平。信息增益排序为每个能够得出结论的模型带来了 +0.26 至 +0.35 的提升，而控制器端的停止机制使 Llama-3.1-8B（其自身从不得出结论）从 0 提升至 0.917。因此，“探测什么”和“何时停止”是两种不同的失败模式，不同的小型模型表现出不同的失败模式。由于 HESP 及其规划器完全在本地硬件上运行，它适用于遥测数据不能离开本地的环境。我们在以下 https URL 发布了所有代码、协议和会话日志。
英文摘要
Security operations centers receive far more alerts than analysts can investigate, and organizations that cannot send their telemetry to hosted models must automate triage with small open-weight LLMs on their own hardware. Current LLM agents leave the investigation procedure to the model, and small local models fail at it: they probe without converging, never commit to a verdict, or dismiss real attacks. In this paper, we present HESP, a controller that holds the investigation procedure outside the model. HESP keeps a ledger of competing explanations, selects read-only probes by expected information gain per cost, accepts only verdicts backed by current evidence, can end an investigation itself, and journals every prediction before its observation. We evaluated HESP in four pre-registered studies with five open-weight models from two families (7B to 72B), totalling 7,272 audited episodes in a controlled triage environment. With likelihood tables counted from LLM-free runs, HESP lifts Qwen2.5-7B from 0.125 to 1.000 verified completion, matching oracle tables. The information-gain ranking adds +0.26 to +0.35 on every model that concludes, and a controller-side stop lifts Llama-3.1-8B, which never concludes on its own, from 0 to 0.917. What to probe and when to stop are therefore separate failures, and different small models exhibit different ones. Because HESP and its planner run entirely on local hardware, it suits environments where telemetry cannot leave the premises. We release all code, protocols, and episode journals at https://github.com/lzwhehe/HESP.
发表机构
- Japan Advanced Institute of Science and Technology (JAIST)（日本北陆先端科学技术大学院大学）
- Liaoning Normal University（辽宁师范大学）
机构由 AI 辅助整理，请以论文原文为准。
