超越语法:安全专家是否信任大语言模型进行网络入侵检测系统规则工程?
Beyond the Syntax: Do Security Experts Trust LLMs for NIDS Rule Engineering?
AI总结:
研究网络威胁下NIDS规则工程瓶颈,通过用户研究评估基于LLM的规则生成框架,揭示语法 - 语义悖论,从业者对其自主能力存疑,且大规模模型生成规则有效,小模型大多无效。
摘要:
随着网络威胁的演变,手动的网络入侵检测系统(NIDS)规则工程成为关键运营瓶颈。大语言模型(LLMs)虽有望自动化此过程,但其生成可用于生产的规则的能力尚未得到验证。本文对基于LLM的NIDS规则工程进行了以人为中心的研究,构建了一个有依据的生成框架,并通过对10位领域专家的用户研究进行评估。评估揭示了语法 - 语义悖论:LLMs生成的规则语法正确,但专家发现由于特异性低和12%的逻辑幻觉情况,它们仅部分可部署。该系统获得了67的SUS评分,但从业者对其自主能力仍持怀疑态度,将LLMs视为起草和验证的支持工具而非独立生成器。最后,统计分析表明,大规模模型(≥70B)始终能生成语法有效的规则,而小模型(≤4B)在IDS规则生成方面大多无效。
英文摘要:
As network threats evolve, manual NIDS rule engineering has become a critical operational bottleneck. While Large Language Models (LLMs) show promise for automating this process, their ability to produce production-ready rules remains unvalidated. This paper presents a human-centered investigation into LLM-based NIDS rule engineering, formalizing a grounded generation framework and evaluating it through a user study with 10 domain experts. Our evaluation reveals a syntax-semantics paradox: although LLMs generate syntactically correct rules, experts find them only partially deployable due to low specificity and logic hallucinations in 12% of cases. While the system received a favorable SUS score of 67, practitioners remain skeptical of its autonomous capabilities, viewing LLMs as support tools for drafting and verification rather than independent generators. Finally, our statistical analysis indicates that while large-scale models ($\geq 70B$) consistently produce syntactically valid rules, small models ($\leq 4B$) are largely ineffective for IDS rule generation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。